[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:10 dist-test-slave-h6n0 krb5kdc[11079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:10 dist-test-slave-h6n0 krb5kdc[11079](info): set up 2 sockets
Aug 07 08:37:10 dist-test-slave-h6n0 krb5kdc[11079](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 08:37:11.477809 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:11 dist-test-slave-h6n0 krb5kdc[11079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:11.617337 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35071
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:35071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:11.732901 11099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:11.733199 11099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:11.733232 11099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:11.734722 11099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:11.734771 11099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:11.734787 11099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:11.734802 11099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:11.734817 11099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:11.736975 11099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:35071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:11.737284 11099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:11.737558 11099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:11.747393 11106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:11.747408 11108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:11.747426 11104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:11.748657 11099 server_base.cc:1034] running on GCE node
I20260807 08:37:11.748986 11099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:11.749511 11099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:11.750756 11099 hybrid_clock.cc:648] HybridClock initialized: now 1786091831750703 us; error 84 us; skew 500 ppm
Aug 07 08:37:11 dist-test-slave-h6n0 krb5kdc[11079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091831, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:11.754698 11099 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:11.755402 11099 webserver.cc:460] Webserver started at http://127.10.207.62:33213/ using document root <none> and password file <none>
I20260807 08:37:11.755753 11099 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:11.755841 11099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:11.756023 11099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:11.757912 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "f2e26d60a1bb4d098e0969dce42f3661"
format_stamp: "Formatted at 2026-08-07 08:37:11 on dist-test-slave-h6n0"
server_key: "957ad104bc9a24f2ad53f097095fa407"
server_key_iv: "09482cb934172538ec16acb6b33ea39c"
server_key_version: "encryptionkey@0"
I20260807 08:37:11.758741 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "f2e26d60a1bb4d098e0969dce42f3661"
format_stamp: "Formatted at 2026-08-07 08:37:11 on dist-test-slave-h6n0"
server_key: "957ad104bc9a24f2ad53f097095fa407"
server_key_iv: "09482cb934172538ec16acb6b33ea39c"
server_key_version: "encryptionkey@0"
I20260807 08:37:11.760875 11099 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 08:37:11.762104 11126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:11.762403 11099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 08:37:11.762528 11099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "f2e26d60a1bb4d098e0969dce42f3661"
format_stamp: "Formatted at 2026-08-07 08:37:11 on dist-test-slave-h6n0"
server_key: "957ad104bc9a24f2ad53f097095fa407"
server_key_iv: "09482cb934172538ec16acb6b33ea39c"
server_key_version: "encryptionkey@0"
I20260807 08:37:11.762740 11099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:11.779693 11099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:11.782644 11099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:11.782859 11099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:11.795409 11099 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:35071
I20260807 08:37:11.795943 11099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:11.796320 11203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:35071 every 8 connection(s)
I20260807 08:37:11.797318 11206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:11.800653 11206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661: Bootstrap starting.
I20260807 08:37:11.801403 11206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:11.801700 11206 log.cc:824] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:11.802868 11206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661: No bootstrap required, opened a new log
I20260807 08:37:11.804464 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11099
I20260807 08:37:11.804596 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:11.805106 11206 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e26d60a1bb4d098e0969dce42f3661" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35071 } }
I20260807 08:37:11.805316 11206 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:11.805348 11206 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e26d60a1bb4d098e0969dce42f3661, State: Initialized, Role: FOLLOWER
I20260807 08:37:11.805464 11068 external_mini_cluster.cc:1428] Setting key bf50fb2e96b00ed88779dabd23758e2d
I20260807 08:37:11.805557 11206 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e26d60a1bb4d098e0969dce42f3661" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35071 } }
I20260807 08:37:11.805661 11206 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:11.805713 11206 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:11.805761 11206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:11.806414 11206 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e26d60a1bb4d098e0969dce42f3661" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35071 } }
I20260807 08:37:11.806603 11206 leader_election.cc:302] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2e26d60a1bb4d098e0969dce42f3661; no voters:
I20260807 08:37:11.806805 11206 leader_election.cc:288] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:11.807060 11213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:11.807137 11206 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:11.810020 11213 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [term 1 LEADER]: Becoming Leader. State: Replica: f2e26d60a1bb4d098e0969dce42f3661, State: Running, Role: LEADER
I20260807 08:37:11.810174 11213 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e26d60a1bb4d098e0969dce42f3661" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35071 } }
I20260807 08:37:11.811124 11214 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2e26d60a1bb4d098e0969dce42f3661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e26d60a1bb4d098e0969dce42f3661" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35071 } } }
I20260807 08:37:11.811239 11214 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:11.812232 11216 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2e26d60a1bb4d098e0969dce42f3661. Latest consensus state: current_term: 1 leader_uuid: "f2e26d60a1bb4d098e0969dce42f3661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e26d60a1bb4d098e0969dce42f3661" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35071 } } }
I20260807 08:37:11.812364 11216 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:11.812304 11226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:11.813618 11226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:11.816444 11226 catalog_manager.cc:1347] Generated new cluster ID: f885a98ae8d54cb4ae764568c444f726
I20260807 08:37:11.817564 11226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:37:11 dist-test-slave-h6n0 krb5kdc[11079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:11.827556 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:11.806565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.10.207.62:35071)
0807 08:37:11.806910 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:11.806923 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:11.807519 (+ 596us) server_negotiation.cc:407] Connection header received
0807 08:37:11.809845 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:11.809852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:11.809927 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:11.810132 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:11.813213 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:11.815017 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:11.816595 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:11.817171 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:11.817301 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:11.821685 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:11.821690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:11.821690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:11.821701 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:11.824860 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:11.825300 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:11.825303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:11.825303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:11.825427 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:11.825681 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:11.825682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:11.825682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:11.826026 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:11.826283 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:11.827064 (+ 781us) server_negotiation.cc:299] Negotiation successful
0807 08:37:11.827198 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":177,"threads_started":1}
I20260807 08:37:11.841080 11226 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:11.842108 11226 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:11.852350 11226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2e26d60a1bb4d098e0969dce42f3661: Generated new TSK 0
I20260807 08:37:11.852581 11226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:11.882800 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35071
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:11.982584 11240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:11.982796 11240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:11.982822 11240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:11.984017 11240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:11.984067 11240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:11.984094 11240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:11.985559 11240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35071
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:11.985826 11240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:11.986112 11240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:11.992980 11249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:11.993019 11251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:11.993402 11253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:11.994061 11240 server_base.cc:1034] running on GCE node
I20260807 08:37:11.994288 11240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:11.994508 11240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:11.995709 11240 hybrid_clock.cc:648] HybridClock initialized: now 1786091831995699 us; error 30 us; skew 500 ppm
Aug 07 08:37:11 dist-test-slave-h6n0 krb5kdc[11079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091831, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:11.998318 11240 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:11.998974 11240 webserver.cc:460] Webserver started at http://127.10.207.1:40511/ using document root <none> and password file <none>
I20260807 08:37:11.999193 11240 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:11.999238 11240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:11.999325 11240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:12.000347 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "d1bc3c40c4304705bb7def7bbb05a620"
format_stamp: "Formatted at 2026-08-07 08:37:11 on dist-test-slave-h6n0"
server_key: "7afde3743c367a4f116b92fe18d59ca2"
server_key_iv: "4b9415719d5ff1d8738c3b8c2d1cb881"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.000656 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "d1bc3c40c4304705bb7def7bbb05a620"
format_stamp: "Formatted at 2026-08-07 08:37:11 on dist-test-slave-h6n0"
server_key: "7afde3743c367a4f116b92fe18d59ca2"
server_key_iv: "4b9415719d5ff1d8738c3b8c2d1cb881"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.002108 11240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:12.002990 11261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.003286 11240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:12.003364 11240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "d1bc3c40c4304705bb7def7bbb05a620"
format_stamp: "Formatted at 2026-08-07 08:37:11 on dist-test-slave-h6n0"
server_key: "7afde3743c367a4f116b92fe18d59ca2"
server_key_iv: "4b9415719d5ff1d8738c3b8c2d1cb881"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.003463 11240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:12.019773 11240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:12.022053 11240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:12.022203 11240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:12.022521 11240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:12.023001 11240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:12.023052 11240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.023074 11240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:12.023087 11240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.032611 11240 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:37637
I20260807 08:37:12.032703 11417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:37637 every 8 connection(s)
I20260807 08:37:12.033039 11240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:12.037997 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11240
I20260807 08:37:12.038084 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:12.038278 11068 external_mini_cluster.cc:1428] Setting key 50d7c95e161c50653b41b8d432ffb688
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091831, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:12.045558 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.034102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:43901 (local address 127.10.207.62:35071)
0807 08:37:12.034233 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:12.034237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:12.034407 (+ 170us) server_negotiation.cc:407] Connection header received
0807 08:37:12.035110 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:12.035111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:12.035125 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:12.035208 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:12.036504 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.037028 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.038231 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.038606 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.038711 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.041672 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:12.041674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:12.041674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:12.041685 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:12.043399 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.043875 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.043876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.043877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.043943 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.044433 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.044435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.044436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.044634 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:12.044739 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:12.045245 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 08:37:12.045383 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 08:37:12.045644 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.033983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35071 (local address 127.10.207.1:43901)
0807 08:37:12.034241 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:12.034291 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:12.034948 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:12.035310 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:12.035315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:12.036125 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:12.036378 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:12.036384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.037133 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.037134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:12.038094 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:12.038098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.038758 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.038759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:12.038921 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.039685 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:12.039696 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:12.041512 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:12.043517 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:12.043526 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:12.043529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:12.043789 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:12.044017 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:12.044018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:12.044018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:12.044264 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:12.044762 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:12.044770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:12.044937 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 08:37:12.045202 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 08:37:12.045430 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":169,"threads_started":1}
I20260807 08:37:12.046489 11418 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35071
I20260807 08:37:12.046662 11418 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:12.046945 11418 heartbeater.cc:511] Master 127.10.207.62:35071 requested a full tablet report, sending...
I20260807 08:37:12.047701 11153 ts_manager.cc:194] Registered new tserver with Master: d1bc3c40c4304705bb7def7bbb05a620 (127.10.207.1:37637)
I20260807 08:37:12.048592 11153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:43901
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:12.069765 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35071
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:12.166842 11429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:12.167084 11429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:12.167117 11429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:12.168452 11429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:12.168502 11429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:12.168531 11429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:12.170212 11429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35071
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:12.170519 11429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:12.170871 11429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:12.176899 11439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:12.176793 11434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:12.176795 11435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:12.177270 11429 server_base.cc:1034] running on GCE node
I20260807 08:37:12.177635 11429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:12.178021 11429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:12.179211 11429 hybrid_clock.cc:648] HybridClock initialized: now 1786091832179186 us; error 42 us; skew 500 ppm
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:12.182924 11429 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:12.183573 11429 webserver.cc:460] Webserver started at http://127.10.207.2:38129/ using document root <none> and password file <none>
I20260807 08:37:12.183921 11429 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:12.184000 11429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:12.184170 11429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:12.185427 11429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "1a92b29191244716a7118742a5698312"
format_stamp: "Formatted at 2026-08-07 08:37:12 on dist-test-slave-h6n0"
server_key: "4cc89217e516d60580f55676800bb7ed"
server_key_iv: "d27b94de752de84b009c05618d5625d0"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.185880 11429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "1a92b29191244716a7118742a5698312"
format_stamp: "Formatted at 2026-08-07 08:37:12 on dist-test-slave-h6n0"
server_key: "4cc89217e516d60580f55676800bb7ed"
server_key_iv: "d27b94de752de84b009c05618d5625d0"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.187944 11429 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 08:37:12.188984 11452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.189272 11429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:12.189395 11429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "1a92b29191244716a7118742a5698312"
format_stamp: "Formatted at 2026-08-07 08:37:12 on dist-test-slave-h6n0"
server_key: "4cc89217e516d60580f55676800bb7ed"
server_key_iv: "d27b94de752de84b009c05618d5625d0"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.189476 11429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:12.198032 11429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:12.200500 11429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:12.200677 11429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:12.201012 11429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:12.201432 11429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:12.201474 11429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.201531 11429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:12.201560 11429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.212929 11429 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:34109
I20260807 08:37:12.213023 11607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:34109 every 8 connection(s)
I20260807 08:37:12.213433 11429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:12.216862 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11429
I20260807 08:37:12.216969 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:12.217139 11068 external_mini_cluster.cc:1428] Setting key 66e2b83dcf3cfc2faadf7c5caa219dc7
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:12.225257 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.214293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:48477 (local address 127.10.207.62:35071)
0807 08:37:12.214352 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:12.214355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:12.214701 (+ 346us) server_negotiation.cc:407] Connection header received
0807 08:37:12.215284 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:12.215285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:12.215297 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:12.215358 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:12.216403 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.217085 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.218469 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.218953 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.219075 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.221688 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:12.221690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:12.221690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:12.221699 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:12.223633 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.224109 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.224110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.224110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.224178 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.224455 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.224456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.224457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.224616 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:12.224709 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:12.225014 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 08:37:12.225121 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:37:12.225370 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.214271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35071 (local address 127.10.207.2:48477)
0807 08:37:12.214484 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:12.214574 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:12.215184 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:12.215423 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:12.215427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:12.216057 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:12.216302 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:12.216307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.217223 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.217224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:12.218328 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:12.218332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.219083 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.219084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:12.219289 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.219950 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:12.219961 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:12.221566 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:12.223751 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:12.223759 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:12.223761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:12.224011 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:12.224263 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:12.224264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:12.224264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:12.224375 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:12.224703 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:12.224709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:12.224855 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 08:37:12.224956 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 08:37:12.225119 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":131,"threads_started":1}
I20260807 08:37:12.225955 11608 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35071
I20260807 08:37:12.226082 11608 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:12.226353 11608 heartbeater.cc:511] Master 127.10.207.62:35071 requested a full tablet report, sending...
I20260807 08:37:12.226917 11153 ts_manager.cc:194] Registered new tserver with Master: 1a92b29191244716a7118742a5698312 (127.10.207.2:34109)
I20260807 08:37:12.227361 11153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:48477
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:12.247892 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35071
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:12.343585 11615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:12.343812 11615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:12.343849 11615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:12.345444 11615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:12.345528 11615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:12.345580 11615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:12.347407 11615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35071
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:12.347661 11615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:12.347894 11615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:12.353279 11635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:12.353299 11628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:12.353447 11632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:12.353957 11615 server_base.cc:1034] running on GCE node
I20260807 08:37:12.354206 11615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:12.354458 11615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:12.355688 11615 hybrid_clock.cc:648] HybridClock initialized: now 1786091832355668 us; error 38 us; skew 500 ppm
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:12.358695 11615 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:12.359205 11615 webserver.cc:460] Webserver started at http://127.10.207.3:33295/ using document root <none> and password file <none>
I20260807 08:37:12.359448 11615 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:12.359520 11615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:12.359663 11615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:12.360687 11615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "ee80f93fbfb0446bae0c55509e938530"
format_stamp: "Formatted at 2026-08-07 08:37:12 on dist-test-slave-h6n0"
server_key: "dc154e9def8f387110e1c2f0b79dc759"
server_key_iv: "2eaf86bc981010244adbbb05b235c707"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.361065 11615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "ee80f93fbfb0446bae0c55509e938530"
format_stamp: "Formatted at 2026-08-07 08:37:12 on dist-test-slave-h6n0"
server_key: "dc154e9def8f387110e1c2f0b79dc759"
server_key_iv: "2eaf86bc981010244adbbb05b235c707"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.362682 11615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:12.363888 11646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.364071 11615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:12.364189 11615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "ee80f93fbfb0446bae0c55509e938530"
format_stamp: "Formatted at 2026-08-07 08:37:12 on dist-test-slave-h6n0"
server_key: "dc154e9def8f387110e1c2f0b79dc759"
server_key_iv: "2eaf86bc981010244adbbb05b235c707"
server_key_version: "encryptionkey@0"
I20260807 08:37:12.364277 11615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:12.373667 11615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:12.375859 11615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:12.376046 11615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:12.376286 11615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:12.376612 11615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:12.376675 11615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.376721 11615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:12.376755 11615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:12.386423 11615 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:39911
I20260807 08:37:12.386482 11806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:39911 every 8 connection(s)
I20260807 08:37:12.386804 11615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:12.395430 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11615
I20260807 08:37:12.395550 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:12.395812 11068 external_mini_cluster.cc:1428] Setting key f63f64b7c5a5125b3acbe8da9db7ed73
I20260807 08:37:12.398489 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.387757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:48285 (local address 127.10.207.62:35071)
0807 08:37:12.387920 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:12.387924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:12.388177 (+ 253us) server_negotiation.cc:407] Connection header received
0807 08:37:12.388825 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:12.388826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:12.388842 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:12.388907 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:12.390061 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.390706 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.391808 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.392021 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.392133 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.394736 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:12.394738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:12.394739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:12.394747 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:12.396617 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.397418 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.397419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.397419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.397494 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.397781 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.397782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.397782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.397905 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:12.397965 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:12.398312 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 08:37:12.398397 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 08:37:12.398654 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.387661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35071 (local address 127.10.207.3:48285)
0807 08:37:12.387953 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:12.388012 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:12.388690 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:12.388990 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:12.388995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:12.389701 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:12.389933 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:12.389938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.390809 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.390810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:12.391695 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:12.391698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.392109 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.392110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:12.392259 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.392979 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:12.392989 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:12.394631 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:12.396922 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:12.396931 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:12.396934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:12.397320 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:12.397593 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:12.397594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:12.397594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:12.397707 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:12.397984 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:12.397989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:12.398147 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 08:37:12.398246 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:37:12.398404 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260807 08:37:12.399237 11807 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35071
I20260807 08:37:12.399348 11807 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:12.399529 11807 heartbeater.cc:511] Master 127.10.207.62:35071 requested a full tablet report, sending...
I20260807 08:37:12.399976 11153 ts_manager.cc:194] Registered new tserver with Master: ee80f93fbfb0446bae0c55509e938530 (127.10.207.3:39911)
I20260807 08:37:12.400382 11153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:48285
I20260807 08:37:12.408879 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:12.439769 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.430206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.10.207.1:37637)
0807 08:37:12.430485 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:12.430491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:12.430505 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:37:12.430606 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:12.430609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:12.430627 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:12.430721 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:12.431398 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.432167 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.432964 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.433291 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.433356 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.436166 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:12.436184 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:12.436184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:12.436192 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:12.438045 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.438463 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.438467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.438468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.438576 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.438942 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.438943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.438943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.439201 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:12.439481 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:12.439532 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 08:37:12.439630 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":122,"threads_started":1}
W20260807 08:37:12.440260 11297 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45600
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:12.470221 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:12.461099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.10.207.1:37637)
0807 08:37:12.461227 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:12.461230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:12.461359 (+ 129us) server_negotiation.cc:407] Connection header received
0807 08:37:12.461400 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:12.461401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:12.461417 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:12.461499 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:12.462183 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.462758 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.463586 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:12.463790 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:12.463852 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:12.466632 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:12.466634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:12.466635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:12.466649 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:12.468583 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.469203 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.469205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.469206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.469352 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:12.469635 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:12.469637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:12.469637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:12.469747 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:12.469869 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:12.470013 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 08:37:12.470108 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 08:37:12.471117 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11240
I20260807 08:37:12.490831 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11429
I20260807 08:37:12.506479 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11615
I20260807 08:37:12.519519 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11099
2026-08-07T08:37:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2384 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11825](info): set up 2 sockets
Aug 07 08:37:12 dist-test-slave-h6n0 krb5kdc[11825](info): commencing operation
krb5kdc: starting...
W20260807 08:37:13.797821 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:13 dist-test-slave-h6n0 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:13.937362 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44325
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:44325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:14.021680 11843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:14.021884 11843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:14.021907 11843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:14.023132 11843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:14.023190 11843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:14.023204 11843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:14.023217 11843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:14.023231 11843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:14.024700 11843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:44325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:14.024947 11843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:14.025200 11843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:14.032361 11855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:14.032379 11852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:14.032565 11843 server_base.cc:1034] running on GCE node
W20260807 08:37:14.032392 11853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:14.032974 11843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:14.033443 11843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:14.034631 11843 hybrid_clock.cc:648] HybridClock initialized: now 1786091834034607 us; error 40 us; skew 500 ppm
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091834, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:14.037859 11843 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:14.038484 11843 webserver.cc:460] Webserver started at http://127.10.207.62:42013/ using document root <none> and password file <none>
I20260807 08:37:14.038806 11843 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:14.038890 11843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:14.039059 11843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:14.040290 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "9ad4c78de66e4aaaa375401cf9ca8565"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "fc1568d44d7fdc13d339632dc796c72d"
server_key_iv: "f261abfe1d0815136923ed288d31feef"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.040719 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "9ad4c78de66e4aaaa375401cf9ca8565"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "fc1568d44d7fdc13d339632dc796c72d"
server_key_iv: "f261abfe1d0815136923ed288d31feef"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.042323 11843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:14.043370 11865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.043684 11843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:14.043814 11843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "9ad4c78de66e4aaaa375401cf9ca8565"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "fc1568d44d7fdc13d339632dc796c72d"
server_key_iv: "f261abfe1d0815136923ed288d31feef"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.043900 11843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:14.067565 11843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:14.070272 11843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:14.070461 11843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:14.075707 11843 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:44325
I20260807 08:37:14.075766 11936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:44325 every 8 connection(s)
I20260807 08:37:14.076225 11843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:14.077385 11937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:14.080507 11937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565: Bootstrap starting.
I20260807 08:37:14.081141 11937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:14.081405 11937 log.cc:824] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:14.082305 11937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565: No bootstrap required, opened a new log
I20260807 08:37:14.084239 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11843
I20260807 08:37:14.084082 11937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44325 } }
I20260807 08:37:14.084378 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:14.084362 11937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:14.084479 11937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ad4c78de66e4aaaa375401cf9ca8565, State: Initialized, Role: FOLLOWER
I20260807 08:37:14.084601 11068 external_mini_cluster.cc:1428] Setting key d63f42fe6755f639f9134907edbced07
I20260807 08:37:14.084641 11937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44325 } }
I20260807 08:37:14.084769 11937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:14.084813 11937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:14.084873 11937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:14.086117 11937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44325 } }
I20260807 08:37:14.086249 11937 leader_election.cc:302] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ad4c78de66e4aaaa375401cf9ca8565; no voters:
I20260807 08:37:14.086442 11937 leader_election.cc:288] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:14.086512 11946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:14.086717 11946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [term 1 LEADER]: Becoming Leader. State: Replica: 9ad4c78de66e4aaaa375401cf9ca8565, State: Running, Role: LEADER
I20260807 08:37:14.086776 11937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:14.086844 11946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44325 } }
I20260807 08:37:14.087258 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ad4c78de66e4aaaa375401cf9ca8565. Latest consensus state: current_term: 1 leader_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44325 } } }
I20260807 08:37:14.087349 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:14.087271 11948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad4c78de66e4aaaa375401cf9ca8565" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44325 } } }
I20260807 08:37:14.087901 11948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:14.088096 11952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:14.088698 11952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:14.090701 11952 catalog_manager.cc:1347] Generated new cluster ID: 8f3029dfa4264d3589d1b77bc06e57a3
I20260807 08:37:14.090771 11952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:14.096720 11952 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:14.099363 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.085383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.10.207.62:44325)
0807 08:37:14.085682 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.085692 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.085725 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:37:14.085853 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.085860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.085911 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.086063 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:14.087038 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.088827 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.090043 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.090763 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.090856 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.094272 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:14.094278 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:14.094278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:14.094295 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:14.096781 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.097302 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.097306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.097306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.097427 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.097868 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.097870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.097870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.098228 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:14.098384 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.098756 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.099071 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":179,"threads_started":1}
I20260807 08:37:14.099644 11952 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:14.108466 11952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ad4c78de66e4aaaa375401cf9ca8565: Generated new TSK 0
I20260807 08:37:14.108745 11952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:14.139882 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44325
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:14.223232 11977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:14.223428 11977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:14.223452 11977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:14.224653 11977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:14.224696 11977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:14.224723 11977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:14.226164 11977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:14.226397 11977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:14.226675 11977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:14.231797 11983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:14.231812 11984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:14.231846 11989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:14.232442 11977 server_base.cc:1034] running on GCE node
I20260807 08:37:14.232658 11977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:14.232900 11977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:14.234087 11977 hybrid_clock.cc:648] HybridClock initialized: now 1786091834234064 us; error 47 us; skew 500 ppm
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091834, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:14.236685 11977 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:14.237195 11977 webserver.cc:460] Webserver started at http://127.10.207.1:40529/ using document root <none> and password file <none>
I20260807 08:37:14.237377 11977 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:14.237421 11977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:14.237509 11977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:14.238425 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "39311bc5d62e434ebc97c4dffe26511b"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "5af720702656d1b17465b9646b86ca52"
server_key_iv: "dede12d190cbbd00cc8a18743127b367"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.238852 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "39311bc5d62e434ebc97c4dffe26511b"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "5af720702656d1b17465b9646b86ca52"
server_key_iv: "dede12d190cbbd00cc8a18743127b367"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.240522 11977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:14.241586 11998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.241854 11977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:14.241986 11977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "39311bc5d62e434ebc97c4dffe26511b"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "5af720702656d1b17465b9646b86ca52"
server_key_iv: "dede12d190cbbd00cc8a18743127b367"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.242062 11977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:14.260036 11977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:14.262404 11977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:14.262580 11977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:14.262804 11977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:14.263356 11977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:14.263410 11977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.263435 11977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:14.263510 11977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.270599 11977 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:45103
I20260807 08:37:14.270728 12160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:45103 every 8 connection(s)
I20260807 08:37:14.271046 11977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:14.274942 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11977
I20260807 08:37:14.275103 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:14.275385 11068 external_mini_cluster.cc:1428] Setting key 70dd0a5a0c7cfb9b5e4f934e41ace078
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091834, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:14.285148 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.272188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:59361 (local address 127.10.207.62:44325)
0807 08:37:14.272242 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.272245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.272745 (+ 500us) server_negotiation.cc:407] Connection header received
0807 08:37:14.274459 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.274461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.274484 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.274616 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:14.275667 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.276470 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.278279 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.278652 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.278691 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.281568 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:14.281569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:14.281570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:14.281576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:14.283448 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.283958 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.283959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.283960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.284036 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.284321 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.284322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.284322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.284470 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:14.284542 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.284920 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.285039 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:37:14.285262 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.272154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44325 (local address 127.10.207.1:59361)
0807 08:37:14.272515 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.272587 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.274289 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.274727 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.274732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.275186 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:14.275520 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.275527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.276770 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.276771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.278148 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.278152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.278773 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.278774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.278893 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.279484 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:14.279493 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:14.281437 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:14.283578 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:14.283585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:14.283588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:14.283860 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:14.284133 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:14.284133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:14.284134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:14.284245 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:14.284605 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:14.284613 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:14.284761 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.284865 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.285022 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":230,"threads_started":1}
I20260807 08:37:14.285979 12161 heartbeater.cc:347] Connected to a master server at 127.10.207.62:44325
I20260807 08:37:14.286152 12161 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:14.286367 12161 heartbeater.cc:511] Master 127.10.207.62:44325 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
I20260807 08:37:14.287191 11890 ts_manager.cc:194] Registered new tserver with Master: 39311bc5d62e434ebc97c4dffe26511b (127.10.207.1:45103)
I20260807 08:37:14.287850 11890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:59361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:14.305131 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44325
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:14.391093 12167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:14.391310 12167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:14.391350 12167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:14.392473 12167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:14.392544 12167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:14.392590 12167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:14.394074 12167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:14.394330 12167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:14.394605 12167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:14.400126 12173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:14.400313 12167 server_base.cc:1034] running on GCE node
W20260807 08:37:14.400180 12174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:14.400120 12177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:14.400769 12167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:14.401116 12167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:14.402314 12167 hybrid_clock.cc:648] HybridClock initialized: now 1786091834402304 us; error 47 us; skew 500 ppm
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091834, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:14.405193 12167 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:14.405696 12167 webserver.cc:460] Webserver started at http://127.10.207.2:34723/ using document root <none> and password file <none>
I20260807 08:37:14.405941 12167 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:14.406014 12167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:14.406158 12167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:14.407270 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "69c93947d2757717e5437f7c4822888a"
server_key_iv: "3b7fbe5dadfe94ddda85941b8d4c77ec"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.407672 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "69c93947d2757717e5437f7c4822888a"
server_key_iv: "3b7fbe5dadfe94ddda85941b8d4c77ec"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.409176 12167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:37:14.410050 12188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.410329 12167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:14.410451 12167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "69c93947d2757717e5437f7c4822888a"
server_key_iv: "3b7fbe5dadfe94ddda85941b8d4c77ec"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.410583 12167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:14.425583 12167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:14.428360 12167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:14.428566 12167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:14.428889 12167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:14.429266 12167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:14.429309 12167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.429335 12167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:14.429348 12167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.436622 12167 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:42717
I20260807 08:37:14.436811 12346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:42717 every 8 connection(s)
I20260807 08:37:14.437124 12167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:14.441697 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12167
I20260807 08:37:14.441797 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:14.442027 11068 external_mini_cluster.cc:1428] Setting key 43e3136df85f5d3dcf6955566208a2a0
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091834, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:14.449203 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.438055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:42405 (local address 127.10.207.62:44325)
0807 08:37:14.438181 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.438185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.438387 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:37:14.439604 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.439607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.439625 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.439682 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:14.440439 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.441004 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.442257 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.442535 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.442636 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.445754 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:14.445756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:14.445756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:14.445773 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:14.447683 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.448169 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.448170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.448171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.448238 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.448513 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.448514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.448514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.448644 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:14.448723 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.449007 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.449104 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 08:37:14.449425 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.437904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44325 (local address 127.10.207.2:42405)
0807 08:37:14.438210 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.438273 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.439441 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.439769 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.439773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.440094 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:14.440328 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.440333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.441118 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.441119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.442168 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.442171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.442647 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.442649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.442793 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.443506 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:14.443519 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:14.445613 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:14.447782 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:14.447790 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:14.447792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:14.448077 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:14.448310 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:14.448311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:14.448311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:14.448427 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:14.448715 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:14.448723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:14.448869 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.448984 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.449175 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":150,"threads_started":1}
I20260807 08:37:14.449987 12348 heartbeater.cc:347] Connected to a master server at 127.10.207.62:44325
I20260807 08:37:14.450088 12348 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:14.450315 12348 heartbeater.cc:511] Master 127.10.207.62:44325 requested a full tablet report, sending...
I20260807 08:37:14.450785 11890 ts_manager.cc:194] Registered new tserver with Master: bc0e74c8e6c44bfe8054aa7e4ec45bdf (127.10.207.2:42717)
I20260807 08:37:14.451383 11890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:42405
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:14.471645 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44325
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:14.565945 12357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:14.566212 12357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:14.566239 12357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:14.567555 12357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:14.567605 12357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:14.567634 12357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:14.569463 12357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:14.569722 12357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:14.569936 12357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:14.575947 12364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:14.575977 12366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:14.576259 12357 server_base.cc:1034] running on GCE node
W20260807 08:37:14.575958 12368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:14.576676 12357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:14.577028 12357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:14.578238 12357 hybrid_clock.cc:648] HybridClock initialized: now 1786091834578219 us; error 36 us; skew 500 ppm
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091834, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:14.581099 12357 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:14.581555 12357 webserver.cc:460] Webserver started at http://127.10.207.3:41375/ using document root <none> and password file <none>
I20260807 08:37:14.581841 12357 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:14.581895 12357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:14.582052 12357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:14.583328 12357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "2cd37ea1c6c04a549d5be18bc8917d8e"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "2735f600feb01af18bbaedff91f550b7"
server_key_iv: "df815fe9abbf9abff3b3bcefdc017bff"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.583650 12357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd37ea1c6c04a549d5be18bc8917d8e"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "2735f600feb01af18bbaedff91f550b7"
server_key_iv: "df815fe9abbf9abff3b3bcefdc017bff"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.585179 12357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:37:14.585951 12377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.586206 12357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:14.586337 12357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "2cd37ea1c6c04a549d5be18bc8917d8e"
format_stamp: "Formatted at 2026-08-07 08:37:14 on dist-test-slave-h6n0"
server_key: "2735f600feb01af18bbaedff91f550b7"
server_key_iv: "df815fe9abbf9abff3b3bcefdc017bff"
server_key_version: "encryptionkey@0"
I20260807 08:37:14.586421 12357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:14.607380 12357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:14.610191 12357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:14.610402 12357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:14.610705 12357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:14.611137 12357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:14.611204 12357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.611255 12357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:14.611315 12357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:14.619318 12357 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34373
I20260807 08:37:14.619433 12541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34373 every 8 connection(s)
I20260807 08:37:14.619822 12357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:37:14 dist-test-slave-h6n0 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091834, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:14.628789 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12357
I20260807 08:37:14.628922 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:14.629104 11068 external_mini_cluster.cc:1428] Setting key 0d1fdc2ad49a30dba190c7d5bbdf7a9d
I20260807 08:37:14.630832 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.620812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:54021 (local address 127.10.207.62:44325)
0807 08:37:14.620878 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.620881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.621112 (+ 231us) server_negotiation.cc:407] Connection header received
0807 08:37:14.622155 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.622157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.622172 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.622242 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:14.623007 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.623528 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.624580 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.624799 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.624896 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.627348 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:14.627349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:14.627350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:14.627357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:14.629251 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.629841 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.629842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.629842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.629899 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.630186 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.630187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.630187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.630293 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:14.630353 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.630693 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.630761 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:37:14.631173 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.620755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44325 (local address 127.10.207.3:54021)
0807 08:37:14.620975 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.621015 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.622023 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.622332 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.622336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.622679 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:14.622883 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.622888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.623640 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.623642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.624460 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.624463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.624909 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.624911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.625050 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.625601 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:14.625610 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:14.627213 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:14.629379 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:14.629390 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:14.629393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:14.629748 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:14.629979 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:14.629980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:14.629980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:14.630125 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:14.630362 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:14.630367 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:14.630509 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.630668 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.630827 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260807 08:37:14.631886 12542 heartbeater.cc:347] Connected to a master server at 127.10.207.62:44325
I20260807 08:37:14.631999 12542 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:14.632236 12542 heartbeater.cc:511] Master 127.10.207.62:44325 requested a full tablet report, sending...
I20260807 08:37:14.632833 11890 ts_manager.cc:194] Registered new tserver with Master: 2cd37ea1c6c04a549d5be18bc8917d8e (127.10.207.3:34373)
I20260807 08:37:14.633296 11890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:54021
I20260807 08:37:14.641016 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:14.649597 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.642846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36970 (local address 127.10.207.62:44325)
0807 08:37:14.642896 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.642898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.643048 (+ 150us) server_negotiation.cc:407] Connection header received
0807 08:37:14.643121 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.643122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.643130 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.643174 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:14.643806 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.644269 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.645058 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.645234 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.645295 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.646785 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:14.646786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:14.646787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:14.646794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:14.648480 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.648877 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.648878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.648878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.648938 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:14.649150 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:14.649150 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:14.649150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:14.649281 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:14.649360 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.649470 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.649537 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:37:14.652405 11890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:14.654204 11890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:14.662117 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.657566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:42717 (local address 127.0.0.1:42514)
0807 08:37:14.657933 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.657954 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.658032 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.658587 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.658588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.658590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:14.658631 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.658632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.659552 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.659552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.660801 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.660803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.661734 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.661736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.661826 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.661838 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.661933 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.662003 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260807 08:37:14.662110 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.657767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.10.207.2:42717)
0807 08:37:14.657949 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.657955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.658348 (+ 393us) server_negotiation.cc:407] Connection header received
0807 08:37:14.658412 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.658414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.658433 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.658553 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:14.658702 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.659442 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.660913 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.661509 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.661617 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.661688 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.661954 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.662006 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260807 08:37:14.662176 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.657731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.10.207.1:45103)
0807 08:37:14.658047 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.658051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.658064 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:37:14.658108 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.658110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.658132 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.658235 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:14.658483 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.659279 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.660844 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.661608 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.661680 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.661742 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.662057 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.662096 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":92,"threads_started":1}
I20260807 08:37:14.663166 12254 tablet_service.cc:1467] Processing CreateTablet for tablet 0e31e96b96ee401f949581deeffebc42 (DEFAULT_TABLE table=test-table [id=7c48837d591043d49738942ae58eda0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:14.663558 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.657793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34373 (local address 127.0.0.1:46342)
0807 08:37:14.658068 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.658078 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.658132 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.659782 (+ 1650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.659783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.659787 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:14.659866 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.659869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.661030 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.661031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.662226 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.662230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.663291 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.663292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.663358 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.663363 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.663437 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.663481 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260807 08:37:14.663619 12254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e31e96b96ee401f949581deeffebc42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:14.663661 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.658084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.10.207.3:34373)
0807 08:37:14.659615 (+ 1531us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.659619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.659627 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:14.659662 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.659663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.659676 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.659780 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:14.659964 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.660907 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.662342 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.663187 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.663249 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.663287 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.663477 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.663525 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1495,"thread_start_us":121,"threads_started":1}
I20260807 08:37:14.664417 12447 tablet_service.cc:1467] Processing CreateTablet for tablet 0e31e96b96ee401f949581deeffebc42 (DEFAULT_TABLE table=test-table [id=7c48837d591043d49738942ae58eda0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:14.664825 12447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e31e96b96ee401f949581deeffebc42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:14.666795 12564 tablet_bootstrap.cc:492] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf: Bootstrap starting.
I20260807 08:37:14.666918 12557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.657555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:45103 (local address 127.0.0.1:50686)
0807 08:37:14.657751 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.657794 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.657873 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.658246 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.658250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.658254 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:14.658327 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.658329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.659391 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.659392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.660739 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.660741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.661779 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.661780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.661865 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.661870 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.661916 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.661984 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260807 08:37:14.667514 12564 tablet_bootstrap.cc:654] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:14.667629 12073 tablet_service.cc:1467] Processing CreateTablet for tablet 0e31e96b96ee401f949581deeffebc42 (DEFAULT_TABLE table=test-table [id=7c48837d591043d49738942ae58eda0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:14.668067 12073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e31e96b96ee401f949581deeffebc42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:14.668169 12564 log.cc:824] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:14.668601 12567 tablet_bootstrap.cc:492] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e: Bootstrap starting.
I20260807 08:37:14.669214 12564 tablet_bootstrap.cc:492] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf: No bootstrap required, opened a new log
I20260807 08:37:14.669296 12564 ts_tablet_manager.cc:1397] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:14.671365 12567 tablet_bootstrap.cc:654] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:14.671438 12564 raft_consensus.cc:348] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.671690 12564 raft_consensus.cc:374] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:14.671749 12564 raft_consensus.cc:729] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0e74c8e6c44bfe8054aa7e4ec45bdf, State: Initialized, Role: FOLLOWER
I20260807 08:37:14.671895 12564 consensus_queue.cc:260] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.671911 12568 tablet_bootstrap.cc:492] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b: Bootstrap starting.
I20260807 08:37:14.672117 12567 log.cc:824] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:14.672206 12564 ts_tablet_manager.cc:1428] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:14.672263 12348 heartbeater.cc:503] Master 127.10.207.62:44325 was elected leader, sending a full tablet report...
I20260807 08:37:14.672935 12568 tablet_bootstrap.cc:654] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:14.672971 12567 tablet_bootstrap.cc:492] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e: No bootstrap required, opened a new log
I20260807 08:37:14.673076 12567 ts_tablet_manager.cc:1397] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 08:37:14.673633 12568 log.cc:824] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:14.675089 12567 raft_consensus.cc:348] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.675269 12567 raft_consensus.cc:374] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:14.675312 12567 raft_consensus.cc:729] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd37ea1c6c04a549d5be18bc8917d8e, State: Initialized, Role: FOLLOWER
I20260807 08:37:14.675438 12567 consensus_queue.cc:260] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.675474 12568 tablet_bootstrap.cc:492] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b: No bootstrap required, opened a new log
I20260807 08:37:14.675604 12568 ts_tablet_manager.cc:1397] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:37:14.675740 12567 ts_tablet_manager.cc:1428] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:37:14.675805 12542 heartbeater.cc:503] Master 127.10.207.62:44325 was elected leader, sending a full tablet report...
I20260807 08:37:14.677718 12568 raft_consensus.cc:348] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.677882 12568 raft_consensus.cc:374] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:14.677907 12568 raft_consensus.cc:729] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39311bc5d62e434ebc97c4dffe26511b, State: Initialized, Role: FOLLOWER
I20260807 08:37:14.678046 12568 consensus_queue.cc:260] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.678380 12568 ts_tablet_manager.cc:1428] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:14.678416 12161 heartbeater.cc:503] Master 127.10.207.62:44325 was elected leader, sending a full tablet report...
W20260807 08:37:14.687785 12350 tablet.cc:2369] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:14.772042 12162 tablet.cc:2369] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:14.870510 12543 tablet.cc:2369] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:14.975028 12572 raft_consensus.cc:482] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:14.975162 12572 raft_consensus.cc:504] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.975625 12572 leader_election.cc:288] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39311bc5d62e434ebc97c4dffe26511b (127.10.207.1:45103), 2cd37ea1c6c04a549d5be18bc8917d8e (127.10.207.3:34373)
I20260807 08:37:14.979221 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.976228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:47283 (local address 127.10.207.3:34373)
0807 08:37:14.976282 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.976284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.976372 (+ 88us) server_negotiation.cc:407] Connection header received
0807 08:37:14.976457 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.976458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.976470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.976535 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:14.976766 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.977277 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.978423 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.978827 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.978889 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.978897 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.979101 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.979147 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:37:14.979334 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.975824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:45103 (local address 127.10.207.2:54697)
0807 08:37:14.975968 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.975983 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.976070 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.976373 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.976374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.976377 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:14.976468 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.976470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.977293 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.977294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.978157 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.978159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.978981 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.978982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.979089 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.979096 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.979163 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.979218 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260807 08:37:14.979326 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.976005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:54697 (local address 127.10.207.1:45103)
0807 08:37:14.976093 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:14.976096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:14.976113 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:37:14.976164 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:14.976165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:14.976183 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:14.976303 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:14.976579 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.977186 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.978280 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.978844 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.978954 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.978966 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:14.979203 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:37:14.979252 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:37:14.979516 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:14.976110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34373 (local address 127.10.207.2:47283)
0807 08:37:14.976240 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:14.976255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:14.976368 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:14.976572 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:14.976573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:14.976576 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:14.976682 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.976685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.977382 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.977383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.978322 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:14.978325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:14.978924 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:14.978925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:14.979007 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:14.979016 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 08:37:14.979079 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:37:14.979124 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
I20260807 08:37:14.979786 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e31e96b96ee401f949581deeffebc42" candidate_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39311bc5d62e434ebc97c4dffe26511b" is_pre_election: true
I20260807 08:37:14.979979 12102 raft_consensus.cc:2454] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0e74c8e6c44bfe8054aa7e4ec45bdf in term 0.
I20260807 08:37:14.980299 12190 leader_election.cc:302] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39311bc5d62e434ebc97c4dffe26511b, bc0e74c8e6c44bfe8054aa7e4ec45bdf; no voters:
I20260807 08:37:14.980319 12475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e31e96b96ee401f949581deeffebc42" candidate_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" is_pre_election: true
I20260807 08:37:14.980511 12475 raft_consensus.cc:2454] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0e74c8e6c44bfe8054aa7e4ec45bdf in term 0.
I20260807 08:37:14.980518 12572 raft_consensus.cc:2781] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:14.980620 12572 raft_consensus.cc:482] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:14.980660 12572 raft_consensus.cc:3037] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:14.981420 12572 raft_consensus.cc:504] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.981581 12572 leader_election.cc:288] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [CANDIDATE]: Term 1 election: Requested vote from peers 39311bc5d62e434ebc97c4dffe26511b (127.10.207.1:45103), 2cd37ea1c6c04a549d5be18bc8917d8e (127.10.207.3:34373)
I20260807 08:37:14.981804 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e31e96b96ee401f949581deeffebc42" candidate_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39311bc5d62e434ebc97c4dffe26511b"
I20260807 08:37:14.981886 12102 raft_consensus.cc:3037] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:14.981894 12475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e31e96b96ee401f949581deeffebc42" candidate_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e"
I20260807 08:37:14.982010 12475 raft_consensus.cc:3037] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:14.982743 12102 raft_consensus.cc:2454] T 0e31e96b96ee401f949581deeffebc42 P 39311bc5d62e434ebc97c4dffe26511b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0e74c8e6c44bfe8054aa7e4ec45bdf in term 1.
I20260807 08:37:14.982962 12190 leader_election.cc:302] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39311bc5d62e434ebc97c4dffe26511b, bc0e74c8e6c44bfe8054aa7e4ec45bdf; no voters:
I20260807 08:37:14.983120 12572 raft_consensus.cc:2781] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:14.983283 12475 raft_consensus.cc:2454] T 0e31e96b96ee401f949581deeffebc42 P 2cd37ea1c6c04a549d5be18bc8917d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0e74c8e6c44bfe8054aa7e4ec45bdf in term 1.
I20260807 08:37:14.983335 12572 raft_consensus.cc:686] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [term 1 LEADER]: Becoming Leader. State: Replica: bc0e74c8e6c44bfe8054aa7e4ec45bdf, State: Running, Role: LEADER
I20260807 08:37:14.983460 12572 consensus_queue.cc:237] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } }
I20260807 08:37:14.984543 11890 catalog_manager.cc:5495] T 0e31e96b96ee401f949581deeffebc42 P bc0e74c8e6c44bfe8054aa7e4ec45bdf reported cstate change: term changed from 0 to 1, leader changed from <none> to bc0e74c8e6c44bfe8054aa7e4ec45bdf (127.10.207.2). New cstate: current_term: 1 leader_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39311bc5d62e434ebc97c4dffe26511b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc0e74c8e6c44bfe8054aa7e4ec45bdf" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 42717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cd37ea1c6c04a549d5be18bc8917d8e" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34373 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:15.023792 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:15.014965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.10.207.1:45103)
0807 08:37:15.015169 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:15.015172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:15.015285 (+ 113us) server_negotiation.cc:407] Connection header received
0807 08:37:15.015379 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:15.015380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:15.015392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:15.015459 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:15.016090 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:15.016578 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:15.017308 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:15.017513 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:15.017593 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:15.020242 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:15.020260 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:15.020260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:15.020271 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:15.022237 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:15.022782 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:15.022785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:15.022786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:15.022870 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:15.023154 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:15.023155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:15.023156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:15.023335 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:15.023476 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:15.023611 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:37:15.023712 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091835, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091835, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:15.054210 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:15.045636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.10.207.1:45103)
0807 08:37:15.045696 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:15.045698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:15.045743 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:37:15.045898 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:15.045899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:15.045910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:15.045971 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:15.046775 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:15.047396 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:15.048219 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:15.048435 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:15.048498 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:15.051575 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:15.051577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:15.051577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:15.051584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:15.052988 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:15.053368 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:15.053369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:15.053370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:15.053430 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:15.053631 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:15.053632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:15.053632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:15.053743 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:15.053837 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:15.054017 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 08:37:15.054127 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[11825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[11825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:15.082991 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:15.074319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.10.207.1:45103)
0807 08:37:15.074405 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:15.074407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:15.074471 (+ 64us) server_negotiation.cc:407] Connection header received
0807 08:37:15.074626 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:15.074627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:15.074638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:15.074698 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:15.075337 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:15.075892 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:15.076993 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:15.077170 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:15.077227 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:15.080049 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:15.080051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:15.080051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:15.080057 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:15.081459 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:15.081885 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:15.081887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:15.081888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:15.081990 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:15.082284 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:15.082285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:15.082285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:15.082406 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:15.082513 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:15.082797 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 08:37:15.082887 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 08:37:15.083640 12041 tablet_service.cc:2908] Rejecting scan request for tablet 0e31e96b96ee401f949581deeffebc42: Uninitialized: clean time has not yet been initialized
I20260807 08:37:15.084937 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11977
I20260807 08:37:15.106280 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12167
I20260807 08:37:15.116153 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12357
I20260807 08:37:15.127342 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11843
2026-08-07T08:37:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2601 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[12600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[12600](info): set up 2 sockets
Aug 07 08:37:15 dist-test-slave-h6n0 krb5kdc[12600](info): commencing operation
krb5kdc: starting...
W20260807 08:37:16.407107 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:16 dist-test-slave-h6n0 krb5kdc[12600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:16.543447 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41295
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:41295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:16.637984 12618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:16.638296 12618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:16.638337 12618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:16.639672 12618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:16.639755 12618 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:37:16.639777 12618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:16.639799 12618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:16.639827 12618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:16.639860 12618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:16.641496 12618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:41295
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:16.641821 12618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:16.642102 12618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:16.648028 12627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:16.648082 12624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:16.648028 12629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:16.648530 12618 server_base.cc:1034] running on GCE node
I20260807 08:37:16.648839 12618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:16.649206 12618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:16.650400 12618 hybrid_clock.cc:648] HybridClock initialized: now 1786091836650369 us; error 51 us; skew 500 ppm
Aug 07 08:37:16 dist-test-slave-h6n0 krb5kdc[12600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:16.659730 12618 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:16.660336 12618 webserver.cc:460] Webserver started at http://127.10.207.62:45641/ using document root <none> and password file <none>
I20260807 08:37:16.660614 12618 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:16.660688 12618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:16.660854 12618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:16.662128 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "1fa8a1f7704f4ee6b32a08936b1aa172"
format_stamp: "Formatted at 2026-08-07 08:37:16 on dist-test-slave-h6n0"
server_key: "8bb7d05a879b09ab1b540d8b94ca13aa"
server_key_iv: "270b48484f37db5b30d35befa2cac23f"
server_key_version: "encryptionkey@0"
I20260807 08:37:16.662628 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "1fa8a1f7704f4ee6b32a08936b1aa172"
format_stamp: "Formatted at 2026-08-07 08:37:16 on dist-test-slave-h6n0"
server_key: "8bb7d05a879b09ab1b540d8b94ca13aa"
server_key_iv: "270b48484f37db5b30d35befa2cac23f"
server_key_version: "encryptionkey@0"
I20260807 08:37:16.664216 12618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:16.665117 12640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:16.665376 12618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:16.665504 12618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "1fa8a1f7704f4ee6b32a08936b1aa172"
format_stamp: "Formatted at 2026-08-07 08:37:16 on dist-test-slave-h6n0"
server_key: "8bb7d05a879b09ab1b540d8b94ca13aa"
server_key_iv: "270b48484f37db5b30d35befa2cac23f"
server_key_version: "encryptionkey@0"
I20260807 08:37:16.665606 12618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:16.681005 12618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:16.683799 12618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:16.683987 12618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:16.690742 12618 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:41295
I20260807 08:37:16.690845 12723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:41295 every 8 connection(s)
I20260807 08:37:16.691206 12618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:16.691996 12724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:16.694761 12724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172: Bootstrap starting.
I20260807 08:37:16.695446 12724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:16.695749 12724 log.cc:824] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:16.696615 12724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172: No bootstrap required, opened a new log
I20260807 08:37:16.699226 12724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41295 } }
I20260807 08:37:16.699400 12724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:16.699424 12724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa8a1f7704f4ee6b32a08936b1aa172, State: Initialized, Role: FOLLOWER
I20260807 08:37:16.699586 12724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41295 } }
I20260807 08:37:16.699680 12724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:16.699707 12724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:16.699738 12724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:16.700363 12724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41295 } }
I20260807 08:37:16.700383 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12618
I20260807 08:37:16.700465 12724 leader_election.cc:302] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fa8a1f7704f4ee6b32a08936b1aa172; no voters:
I20260807 08:37:16.700508 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:16.700696 12724 leader_election.cc:288] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:16.700753 11068 external_mini_cluster.cc:1428] Setting key a19dfa70adb12381317e27a1bee03980
I20260807 08:37:16.700791 12728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:16.701020 12724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:16.701205 12728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [term 1 LEADER]: Becoming Leader. State: Replica: 1fa8a1f7704f4ee6b32a08936b1aa172, State: Running, Role: LEADER
I20260807 08:37:16.701414 12728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41295 } }
I20260807 08:37:16.702050 12731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fa8a1f7704f4ee6b32a08936b1aa172. Latest consensus state: current_term: 1 leader_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41295 } } }
I20260807 08:37:16.702167 12731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:16.702227 12730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa8a1f7704f4ee6b32a08936b1aa172" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41295 } } }
I20260807 08:37:16.702337 12730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:16.702594 12743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:16.703462 12743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:16.707077 12743 catalog_manager.cc:1347] Generated new cluster ID: 7f59b83c3dc9419d97eb1c091df39609
I20260807 08:37:16.707140 12743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:37:16 dist-test-slave-h6n0 krb5kdc[12600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:16.713119 12743 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:16.713877 12743 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:16.716912 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:16.701419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.10.207.62:41295)
0807 08:37:16.701714 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:16.701726 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:16.701905 (+ 179us) server_negotiation.cc:407] Connection header received
0807 08:37:16.702072 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:16.702083 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:16.702167 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:16.702317 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:16.703430 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.704951 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.705792 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.706482 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.706654 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:16.713090 (+ 6436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:16.713100 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:16.713100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:16.713115 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:16.715219 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:16.715616 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:16.715620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:16.715620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:16.715722 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:16.715940 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:16.715941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:16.715941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:16.716180 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:16.716349 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:16.716455 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 08:37:16.716720 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":89,"threads_started":1}
I20260807 08:37:16.718385 12743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172: Generated new TSK 0
I20260807 08:37:16.718606 12743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:16.757154 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41295
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:16.853646 12759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:16.853937 12759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:16.853983 12759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:16.855300 12759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:16.855382 12759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:16.855420 12759 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:16.855463 12759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:16.857053 12759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:41295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:16.857388 12759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:16.857671 12759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:16.863586 12770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:16.863586 12768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:16.864064 12759 server_base.cc:1034] running on GCE node
W20260807 08:37:16.863610 12772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:16.864447 12759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:16.864786 12759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:16.865984 12759 hybrid_clock.cc:648] HybridClock initialized: now 1786091836865976 us; error 46 us; skew 500 ppm
Aug 07 08:37:16 dist-test-slave-h6n0 krb5kdc[12600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:16.869544 12759 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:16.870208 12759 webserver.cc:460] Webserver started at http://127.10.207.1:36071/ using document root <none> and password file <none>
I20260807 08:37:16.870510 12759 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:16.870676 12759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:16.870798 12759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:16.872002 12759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "7d74056d0df64cf481a786defeb6621d"
format_stamp: "Formatted at 2026-08-07 08:37:16 on dist-test-slave-h6n0"
server_key: "c55986fda7e7ca4b70ae8ec39e27a998"
server_key_iv: "99b043d43e9f4c5417cf39653aee4d82"
server_key_version: "encryptionkey@0"
I20260807 08:37:16.872361 12759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "7d74056d0df64cf481a786defeb6621d"
format_stamp: "Formatted at 2026-08-07 08:37:16 on dist-test-slave-h6n0"
server_key: "c55986fda7e7ca4b70ae8ec39e27a998"
server_key_iv: "99b043d43e9f4c5417cf39653aee4d82"
server_key_version: "encryptionkey@0"
I20260807 08:37:16.873871 12759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:16.874780 12784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:16.875080 12759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:16.875217 12759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "7d74056d0df64cf481a786defeb6621d"
format_stamp: "Formatted at 2026-08-07 08:37:16 on dist-test-slave-h6n0"
server_key: "c55986fda7e7ca4b70ae8ec39e27a998"
server_key_iv: "99b043d43e9f4c5417cf39653aee4d82"
server_key_version: "encryptionkey@0"
I20260807 08:37:16.875308 12759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:16.903584 12759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:16.906091 12759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:16.906385 12759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:16.907075 12759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:16.907153 12759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:16.907213 12759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:16.907241 12759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:16.915557 12759 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:41975
I20260807 08:37:16.915663 12951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:41975 every 8 connection(s)
Aug 07 08:37:16 dist-test-slave-h6n0 krb5kdc[12600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:16.916111 12759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:16.920154 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:16.908327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:36453 (local address 127.10.207.62:41295)
0807 08:37:16.908393 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:16.908397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:16.908673 (+ 276us) server_negotiation.cc:407] Connection header received
0807 08:37:16.909414 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:16.909416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:16.909437 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:16.909515 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:16.910646 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.911582 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.913259 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.913501 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.913602 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:16.916973 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:16.916974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:16.916974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:16.916982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:16.918645 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:16.919126 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:16.919126 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:16.919127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:16.919213 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:16.919486 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:16.919487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:16.919487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:16.919607 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:16.919666 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:16.919967 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 08:37:16.920050 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:37:16.920343 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:16.908128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41295 (local address 127.10.207.1:36453)
0807 08:37:16.908471 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:16.908524 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:16.909258 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:16.909625 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:16.909631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:16.910070 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:16.910511 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:16.910518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.911690 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.911691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:16.913150 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:16.913153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.913616 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.913618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:16.913789 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:16.914510 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:16.914520 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:16.916808 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:16.918754 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:16.918762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:16.918765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:16.919039 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:16.919290 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:16.919290 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:16.919291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:16.919414 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:16.919673 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:16.919678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:16.919821 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 08:37:16.919932 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:37:16.920087 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":116,"threads_started":1}
I20260807 08:37:16.921041 12952 heartbeater.cc:347] Connected to a master server at 127.10.207.62:41295
I20260807 08:37:16.921195 12952 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:16.921463 12952 heartbeater.cc:511] Master 127.10.207.62:41295 requested a full tablet report, sending...
I20260807 08:37:16.922215 12662 ts_manager.cc:194] Registered new tserver with Master: 7d74056d0df64cf481a786defeb6621d (127.10.207.1:41975)
I20260807 08:37:16.923172 12662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:36453
I20260807 08:37:16.924449 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12759
I20260807 08:37:16.924566 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:16.924788 11068 external_mini_cluster.cc:1428] Setting key ef73acd78dcde0615a84a4e9b40d83b2
I20260807 08:37:16.930497 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:16.922403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41295 (local address 127.10.207.1:53725)
0807 08:37:16.922561 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:16.922571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:16.922637 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:16.922777 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:16.922777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:16.922948 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:16.923003 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:16.923004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.924186 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.924188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:16.925229 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:16.925234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.925802 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.925804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:16.925883 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:16.926478 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:16.926483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:16.926913 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:16.929261 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:16.929263 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:16.929263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:16.929533 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:16.929825 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:16.929826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:16.929826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:16.929889 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:16.930218 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:16.930219 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:16.930261 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 08:37:16.930321 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:37:16.930417 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260807 08:37:16.930662 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:16.922468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:53725 (local address 127.10.207.62:41295)
0807 08:37:16.922513 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:16.922515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:16.922656 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:37:16.922683 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:16.922684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:16.922695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:16.922771 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:16.923082 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.923868 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.925380 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:16.925710 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:16.925807 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:16.927013 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:16.927015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:16.927015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:16.927024 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:16.929148 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:16.929650 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:16.929651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:16.929651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:16.929732 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:16.929972 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:16.929973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:16.929973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:16.930138 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:16.930222 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:16.930382 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 08:37:16.930490 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:16.953848 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41295
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:17.042254 12966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:17.042457 12966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:17.042480 12966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:17.043905 12966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:17.044001 12966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:17.044178 12966 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:17.044227 12966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:17.046736 12966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:41295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:17.046998 12966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:17.047345 12966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:17.053387 12977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:17.053414 12980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:17.053401 12976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:17.053740 12966 server_base.cc:1034] running on GCE node
I20260807 08:37:17.054054 12966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:17.054368 12966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:17.055586 12966 hybrid_clock.cc:648] HybridClock initialized: now 1786091837055535 us; error 88 us; skew 500 ppm
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091837, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:17.058915 12966 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:17.059496 12966 webserver.cc:460] Webserver started at http://127.10.207.2:34203/ using document root <none> and password file <none>
I20260807 08:37:17.059743 12966 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:17.059818 12966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:17.059978 12966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:17.061069 12966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "3090408d76ad46159cc8aad858dbe5dd"
format_stamp: "Formatted at 2026-08-07 08:37:17 on dist-test-slave-h6n0"
server_key: "c98bb4e1ebcb9a96cb08e093bad5b217"
server_key_iv: "7ebbd7e185977d6577620868f2dc7873"
server_key_version: "encryptionkey@0"
I20260807 08:37:17.061460 12966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "3090408d76ad46159cc8aad858dbe5dd"
format_stamp: "Formatted at 2026-08-07 08:37:17 on dist-test-slave-h6n0"
server_key: "c98bb4e1ebcb9a96cb08e093bad5b217"
server_key_iv: "7ebbd7e185977d6577620868f2dc7873"
server_key_version: "encryptionkey@0"
I20260807 08:37:17.063118 12966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:17.064432 12988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:17.064797 12966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:17.064917 12966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "3090408d76ad46159cc8aad858dbe5dd"
format_stamp: "Formatted at 2026-08-07 08:37:17 on dist-test-slave-h6n0"
server_key: "c98bb4e1ebcb9a96cb08e093bad5b217"
server_key_iv: "7ebbd7e185977d6577620868f2dc7873"
server_key_version: "encryptionkey@0"
I20260807 08:37:17.065024 12966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:17.087356 12966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:17.090106 12966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:17.090325 12966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:17.091256 12966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:17.091343 12966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:17.091384 12966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:17.091419 12966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091837, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:17.099983 12966 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:43131
I20260807 08:37:17.100106 13155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:43131 every 8 connection(s)
I20260807 08:37:17.100438 12966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:17.103410 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.091594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:51773 (local address 127.10.207.62:41295)
0807 08:37:17.091673 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.091676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.091950 (+ 274us) server_negotiation.cc:407] Connection header received
0807 08:37:17.092548 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.092550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.092569 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.092661 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.093702 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.094522 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.095817 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.096045 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.096131 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.099783 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.099785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.099785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.099795 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.101712 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.102258 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.102259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.102260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.102323 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.102636 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.102637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.102637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.102781 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.102859 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.103200 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.103303 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:37:17.103641 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.091568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41295 (local address 127.10.207.2:51773)
0807 08:37:17.091760 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.091802 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.092415 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.092747 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.092751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.093154 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:17.093581 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.093589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.094702 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.094703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.095651 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.095654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.096160 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.096162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.096377 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.097040 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:17.097053 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:17.099629 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:17.101841 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.101850 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.101852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.102161 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.102406 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.102406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.102407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.102576 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.102862 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:17.102870 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:17.103046 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.103176 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.103377 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260807 08:37:17.104310 13157 heartbeater.cc:347] Connected to a master server at 127.10.207.62:41295
I20260807 08:37:17.104429 13157 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:17.104621 13157 heartbeater.cc:511] Master 127.10.207.62:41295 requested a full tablet report, sending...
I20260807 08:37:17.105077 12664 ts_manager.cc:194] Registered new tserver with Master: 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131)
I20260807 08:37:17.105656 12664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:51773
I20260807 08:37:17.110076 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12966
I20260807 08:37:17.110167 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:17.110416 11068 external_mini_cluster.cc:1428] Setting key e3a19ecbc1e1b0bce122cab990ff983d
I20260807 08:37:17.111552 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.105599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:56387 (local address 127.10.207.62:41295)
0807 08:37:17.105634 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.105636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.105801 (+ 165us) server_negotiation.cc:407] Connection header received
0807 08:37:17.105917 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.105918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.105928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.105978 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.106394 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.106975 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.107713 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.107897 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.107957 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.108831 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.108832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.108832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.108839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.110230 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.110688 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.110689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.110690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.110754 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.110972 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.110973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.110973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.111129 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.111215 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.111377 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.111460 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:37:17.111596 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.105515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41295 (local address 127.10.207.2:56387)
0807 08:37:17.105713 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.105723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.105830 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.106013 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.106014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.106233 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:17.106303 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.106304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.107076 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.107076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.107633 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.107634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.107971 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.107972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.108033 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.108414 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:17.108419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:17.108723 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:17.110326 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.110327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.110327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.110612 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.110836 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.110836 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.110837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.110905 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.111207 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:17.111208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:17.111260 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.111350 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.111482 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:17.139983 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41295
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:17.235502 13172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:17.235765 13172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:17.235791 13172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:17.237104 13172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:17.237151 13172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:17.237172 13172 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:17.237192 13172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:17.238867 13172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:41295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:17.239177 13172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:17.239456 13172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:17.245263 13180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:17.245272 13172 server_base.cc:1034] running on GCE node
W20260807 08:37:17.245244 13179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:17.245245 13182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:17.245828 13172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:17.246127 13172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:17.247283 13172 hybrid_clock.cc:648] HybridClock initialized: now 1786091837247258 us; error 51 us; skew 500 ppm
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091837, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:17.249998 13172 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:17.250475 13172 webserver.cc:460] Webserver started at http://127.10.207.3:40461/ using document root <none> and password file <none>
I20260807 08:37:17.250778 13172 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:17.250865 13172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:17.251029 13172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:17.252328 13172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "3ab64f59afb54d1d99f4c06068b60670"
format_stamp: "Formatted at 2026-08-07 08:37:17 on dist-test-slave-h6n0"
server_key: "b629dffdfdd9a851dfd4f0f1999ea9e0"
server_key_iv: "82baf9e666018583f1e8f5587df4125c"
server_key_version: "encryptionkey@0"
I20260807 08:37:17.252831 13172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "3ab64f59afb54d1d99f4c06068b60670"
format_stamp: "Formatted at 2026-08-07 08:37:17 on dist-test-slave-h6n0"
server_key: "b629dffdfdd9a851dfd4f0f1999ea9e0"
server_key_iv: "82baf9e666018583f1e8f5587df4125c"
server_key_version: "encryptionkey@0"
I20260807 08:37:17.254380 13172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:17.255312 13194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:17.255582 13172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:17.255699 13172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "3ab64f59afb54d1d99f4c06068b60670"
format_stamp: "Formatted at 2026-08-07 08:37:17 on dist-test-slave-h6n0"
server_key: "b629dffdfdd9a851dfd4f0f1999ea9e0"
server_key_iv: "82baf9e666018583f1e8f5587df4125c"
server_key_version: "encryptionkey@0"
I20260807 08:37:17.255784 13172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:17.276041 13172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:17.278635 13172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:17.278829 13172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:17.279454 13172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:17.279529 13172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:17.279575 13172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:17.279603 13172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091837, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:17.287518 13172 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:39999
I20260807 08:37:17.287892 13172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:17.288389 13362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:39999 every 8 connection(s)
I20260807 08:37:17.290822 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.280210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:55061 (local address 127.10.207.62:41295)
0807 08:37:17.280312 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.280315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.280583 (+ 268us) server_negotiation.cc:407] Connection header received
0807 08:37:17.281389 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.281391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.281410 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.281471 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.282219 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.283005 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.284176 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.284486 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.284568 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.287530 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.287532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.287532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.287540 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.289281 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.289714 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.289715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.289716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.289783 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.290070 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.290071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.290072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.290201 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.290279 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.290627 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.290713 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:37:17.291041 13204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.280149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41295 (local address 127.10.207.3:55061)
0807 08:37:17.280408 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.280456 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.281259 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.281556 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.281561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.281875 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:17.282102 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.282107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.283092 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.283093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.284067 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.284070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.284575 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.284576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.284736 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.285342 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:17.285354 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:17.287419 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:17.289372 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.289379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.289382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.289629 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.289861 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.289862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.289863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.289984 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.290282 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:17.290290 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:17.290439 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.290602 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.290788 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":131,"threads_started":1}
I20260807 08:37:17.291677 13363 heartbeater.cc:347] Connected to a master server at 127.10.207.62:41295
I20260807 08:37:17.291790 13363 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:17.292017 13363 heartbeater.cc:511] Master 127.10.207.62:41295 requested a full tablet report, sending...
I20260807 08:37:17.292539 12662 ts_manager.cc:194] Registered new tserver with Master: 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3:39999)
I20260807 08:37:17.293104 12662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:55061
I20260807 08:37:17.297380 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 13172
I20260807 08:37:17.297500 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:17.297688 11068 external_mini_cluster.cc:1428] Setting key 9c03f5d7d7f3827bf5fedadbb3b483ca
I20260807 08:37:17.298828 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:17.299670 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.293021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41295 (local address 127.10.207.3:46567)
0807 08:37:17.293198 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.293212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.293286 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.293601 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.293602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.293776 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:17.293849 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.293851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.294644 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.294646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.295204 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.295206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.295606 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.295607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.295676 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.296182 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:17.296190 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:17.296493 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:17.298522 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.298524 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.298525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.298840 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.299069 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.299070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.299070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.299130 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.299409 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:17.299410 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:17.299448 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.299503 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.299599 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260807 08:37:17.299742 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.293145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:46567 (local address 127.10.207.62:41295)
0807 08:37:17.293190 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.293192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.293464 (+ 272us) server_negotiation.cc:407] Connection header received
0807 08:37:17.293495 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.293496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.293509 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.293584 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.293949 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.294451 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.295312 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.295511 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.295585 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.296606 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.296608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.296608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.296614 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.298418 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.298927 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.298928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.298928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.298998 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.299201 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.299202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.299202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.299339 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.299419 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.299534 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.299638 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:17.328225 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.320092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34952 (local address 127.10.207.62:41295)
0807 08:37:17.320217 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.320219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.320232 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:37:17.320272 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.320274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.320286 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.320350 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.320977 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.321535 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.322514 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.322756 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.322848 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.325374 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.325376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.325376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.325384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.326917 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.327355 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.327356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.327356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.327413 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.327665 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.327666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.327666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.327777 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.327860 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.328081 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.328159 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 08:37:17.330145 12662 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:17.331871 12662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:17.339656 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.334909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41975 (local address 127.0.0.1:34350)
0807 08:37:17.335374 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.335423 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.335508 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.335980 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.335982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.335985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:17.336068 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.336070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.337415 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.337416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.338519 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.338522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.339387 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.339388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.339454 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.339466 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.339532 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.339589 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":59,"thread_start_us":94,"threads_started":1}
I20260807 08:37:17.339715 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.335078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.10.207.1:41975)
0807 08:37:17.335798 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.335803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.335813 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:17.335863 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.335864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.335878 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.335980 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:17.336195 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.337299 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.338646 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.339293 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.339377 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.339436 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.339567 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.339611 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":141,"threads_started":1}
I20260807 08:37:17.340580 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.334955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:39999 (local address 127.0.0.1:54888)
0807 08:37:17.335478 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.335491 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.335554 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.335838 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.335841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.335844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:17.335911 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.335915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.337254 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.337255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.338891 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.338894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.340273 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.340274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.340387 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.340391 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.340460 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.340508 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":82,"thread_start_us":101,"threads_started":1}
I20260807 08:37:17.340626 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.335086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.10.207.3:39999)
0807 08:37:17.335269 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.335274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.335617 (+ 343us) server_negotiation.cc:407] Connection header received
0807 08:37:17.335659 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.335662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.335679 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.335755 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:17.336023 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.337144 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.338999 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.339700 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.339781 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.339836 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.340495 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.340534 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":118,"threads_started":1}
I20260807 08:37:17.340816 12857 tablet_service.cc:1467] Processing CreateTablet for tablet 6aa3b558489345b8970df95816817768 (DEFAULT_TABLE table=test-table [id=ba2715885e4b41e48d875a1205a05f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:17.341126 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.335261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43131 (local address 127.0.0.1:37728)
0807 08:37:17.336082 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.336092 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.336149 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.336368 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.336369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.336371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:17.336418 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.336420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.337388 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.337390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.338901 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.338903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.340832 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.340833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.340907 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.340912 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.340976 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.341049 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":143,"thread_start_us":88,"threads_started":1}
I20260807 08:37:17.341286 12857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aa3b558489345b8970df95816817768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.341385 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.335547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.10.207.2:43131)
0807 08:37:17.335833 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.335837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.336210 (+ 373us) server_negotiation.cc:407] Connection header received
0807 08:37:17.336261 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.336263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.336276 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.336371 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:17.336483 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.337263 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.339887 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.340750 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.340826 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.340878 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.341252 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.341296 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":118,"threads_started":1}
I20260807 08:37:17.341615 13274 tablet_service.cc:1467] Processing CreateTablet for tablet 6aa3b558489345b8970df95816817768 (DEFAULT_TABLE table=test-table [id=ba2715885e4b41e48d875a1205a05f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:17.341961 13274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aa3b558489345b8970df95816817768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.342024 13060 tablet_service.cc:1467] Processing CreateTablet for tablet 6aa3b558489345b8970df95816817768 (DEFAULT_TABLE table=test-table [id=ba2715885e4b41e48d875a1205a05f08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:17.342337 13060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aa3b558489345b8970df95816817768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.345675 13397 tablet_bootstrap.cc:492] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Bootstrap starting.
I20260807 08:37:17.346020 13398 tablet_bootstrap.cc:492] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Bootstrap starting.
I20260807 08:37:17.346808 13398 tablet_bootstrap.cc:654] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.347020 13397 tablet_bootstrap.cc:654] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.347226 13398 log.cc:824] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:17.347394 13397 log.cc:824] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:17.347633 13399 tablet_bootstrap.cc:492] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Bootstrap starting.
I20260807 08:37:17.348198 13397 tablet_bootstrap.cc:492] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: No bootstrap required, opened a new log
I20260807 08:37:17.348295 13397 ts_tablet_manager.cc:1397] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:37:17.348274 13399 tablet_bootstrap.cc:654] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.348608 13399 log.cc:824] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:17.349215 13398 tablet_bootstrap.cc:492] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: No bootstrap required, opened a new log
I20260807 08:37:17.349318 13398 ts_tablet_manager.cc:1397] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:17.349453 13399 tablet_bootstrap.cc:492] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: No bootstrap required, opened a new log
I20260807 08:37:17.349537 13399 ts_tablet_manager.cc:1397] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:17.350335 13397 raft_consensus.cc:348] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.350514 13397 raft_consensus.cc:374] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.350615 13397 raft_consensus.cc:729] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab64f59afb54d1d99f4c06068b60670, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.350754 13397 consensus_queue.cc:260] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.351048 13397 ts_tablet_manager.cc:1428] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:37:17.351172 13363 heartbeater.cc:503] Master 127.10.207.62:41295 was elected leader, sending a full tablet report...
I20260807 08:37:17.351382 13398 raft_consensus.cc:348] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.351604 13398 raft_consensus.cc:374] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.351656 13398 raft_consensus.cc:729] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d74056d0df64cf481a786defeb6621d, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.351617 13399 raft_consensus.cc:348] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.351771 13399 raft_consensus.cc:374] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.351809 13399 raft_consensus.cc:729] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3090408d76ad46159cc8aad858dbe5dd, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.351806 13398 consensus_queue.cc:260] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.351922 13399 consensus_queue.cc:260] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.352154 13398 ts_tablet_manager.cc:1428] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:17.352295 12952 heartbeater.cc:503] Master 127.10.207.62:41295 was elected leader, sending a full tablet report...
I20260807 08:37:17.352334 13399 ts_tablet_manager.cc:1428] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:17.352409 13157 heartbeater.cc:503] Master 127.10.207.62:41295 was elected leader, sending a full tablet report...
I20260807 08:37:17.398463 13405 raft_consensus.cc:482] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:17.398711 13405 raft_consensus.cc:504] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.399143 13405 leader_election.cc:288] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3:39999), 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131)
I20260807 08:37:17.402585 12796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.399377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:39999 (local address 127.10.207.1:49515)
0807 08:37:17.399455 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.399468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.399517 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.399747 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.399749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.399752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:17.399847 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.399849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.400569 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.400570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.401519 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.401522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.402233 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.402235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.402332 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.402338 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.402400 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.402451 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":34}
I20260807 08:37:17.402598 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.399482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:49515 (local address 127.10.207.3:39999)
0807 08:37:17.399555 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.399558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.399572 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:37:17.399612 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.399613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.399626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.399738 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:17.399936 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.400451 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.401696 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.402111 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.402176 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.402183 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.402440 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.402483 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:37:17.402807 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.399695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:41201 (local address 127.10.207.2:43131)
0807 08:37:17.399744 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.399746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.399786 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:37:17.399861 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.399862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.399871 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.399912 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:17.400104 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.400644 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.401783 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.402322 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.402410 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.402421 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.402623 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.402720 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:37:17.402817 13410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.399532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43131 (local address 127.10.207.1:41201)
0807 08:37:17.399730 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.399740 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.399798 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.399932 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.399932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.399934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:17.400028 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.400030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.400731 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.400732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.401678 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.401682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.402441 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.402442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.402521 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.402525 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.402614 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.402705 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":120,"threads_started":1}
I20260807 08:37:17.403086 13300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa3b558489345b8970df95816817768" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab64f59afb54d1d99f4c06068b60670" is_pre_election: true
I20260807 08:37:17.403303 13300 raft_consensus.cc:2454] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 0.
I20260807 08:37:17.403321 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa3b558489345b8970df95816817768" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3090408d76ad46159cc8aad858dbe5dd" is_pre_election: true
I20260807 08:37:17.403527 13089 raft_consensus.cc:2454] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 0.
I20260807 08:37:17.403623 12786 leader_election.cc:302] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab64f59afb54d1d99f4c06068b60670, 7d74056d0df64cf481a786defeb6621d; no voters:
I20260807 08:37:17.403779 13405 raft_consensus.cc:2781] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:17.403854 13405 raft_consensus.cc:482] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:17.403882 13405 raft_consensus.cc:3037] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.404866 13405 raft_consensus.cc:504] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.405021 13405 leader_election.cc:288] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 election: Requested vote from peers 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3:39999), 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131)
I20260807 08:37:17.405222 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa3b558489345b8970df95816817768" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3090408d76ad46159cc8aad858dbe5dd"
I20260807 08:37:17.405192 13300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa3b558489345b8970df95816817768" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab64f59afb54d1d99f4c06068b60670"
I20260807 08:37:17.405300 13089 raft_consensus.cc:3037] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.405303 13300 raft_consensus.cc:3037] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.406296 13300 raft_consensus.cc:2454] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 1.
I20260807 08:37:17.406296 13089 raft_consensus.cc:2454] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 1.
I20260807 08:37:17.406479 12786 leader_election.cc:302] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3090408d76ad46159cc8aad858dbe5dd, 7d74056d0df64cf481a786defeb6621d; no voters:
I20260807 08:37:17.406616 13405 raft_consensus.cc:2781] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:17.406805 13405 raft_consensus.cc:686] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 1 LEADER]: Becoming Leader. State: Replica: 7d74056d0df64cf481a786defeb6621d, State: Running, Role: LEADER
I20260807 08:37:17.406946 13405 consensus_queue.cc:237] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } }
I20260807 08:37:17.408033 12664 catalog_manager.cc:5495] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d74056d0df64cf481a786defeb6621d (127.10.207.1). New cstate: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:17.416666 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.414000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34360 (local address 127.10.207.1:41975)
0807 08:37:17.414059 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.414061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.414072 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:17.414176 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.414177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.414193 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.414253 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:17.414407 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.414900 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.415776 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.415950 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.416011 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.416291 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.416381 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.416445 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.416533 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.416550 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 08:37:17.417292 12955 tablet.cc:2369] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:17.418951 13089 raft_consensus.cc:1261] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Refusing update from remote peer 7d74056d0df64cf481a786defeb6621d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:17.418951 13300 raft_consensus.cc:1261] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Refusing update from remote peer 7d74056d0df64cf481a786defeb6621d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:17.419410 13412 consensus_queue.cc:1035] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:17.419559 13405 consensus_queue.cc:1035] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:17.422185 13415 mvcc.cc:205] Tried to move back new op lower bound from 7315832166064582656 to 7315832166019723264. Current Snapshot: MvccSnapshot[applied={T|T < 7315832166064582656}]
I20260807 08:37:17.426246 12664 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:17.426400 12664 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:17.427559 12664 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172: Sending DeleteTablet for 3 replicas of tablet 6aa3b558489345b8970df95816817768
I20260807 08:37:17.428176 12857 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aa3b558489345b8970df95816817768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:17 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:34350
I20260807 08:37:17.428192 13060 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aa3b558489345b8970df95816817768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:17 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37728
I20260807 08:37:17.428469 13274 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aa3b558489345b8970df95816817768 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:17 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:54888
I20260807 08:37:17.428669 13431 tablet_replica.cc:330] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: stopping tablet replica
I20260807 08:37:17.428792 13431 raft_consensus.cc:2229] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:17.428802 12664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:37:17.428934 13431 raft_consensus.cc:2258] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 08:37:17.428973 12664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:17.428905 13432 tablet_replica.cc:330] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: stopping tablet replica
I20260807 08:37:17.429761 13432 raft_consensus.cc:2229] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:17.430123 13431 ts_tablet_manager.cc:1905] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:17.430270 13432 raft_consensus.cc:2258] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:17.430444 13060 tablet_service.cc:1467] Processing CreateTablet for tablet 4e864d52724d499a820a2dd2d25c7e1d (DEFAULT_TABLE table=test-table [id=d0c3c78312174fba802a88dab1160418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:17.430631 13060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e864d52724d499a820a2dd2d25c7e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.432394 13431 ts_tablet_manager.cc:1918] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:17.432505 13431 log.cc:1196] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/6aa3b558489345b8970df95816817768
I20260807 08:37:17.432710 13399 tablet_bootstrap.cc:492] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Bootstrap starting.
I20260807 08:37:17.433039 13431 ts_tablet_manager.cc:1939] T 6aa3b558489345b8970df95816817768 P 3090408d76ad46159cc8aad858dbe5dd: Deleting consensus metadata
I20260807 08:37:17.433120 13274 tablet_service.cc:1467] Processing CreateTablet for tablet 4e864d52724d499a820a2dd2d25c7e1d (DEFAULT_TABLE table=test-table [id=d0c3c78312174fba802a88dab1160418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:17.433202 13399 tablet_bootstrap.cc:654] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.433247 13274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e864d52724d499a820a2dd2d25c7e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.434044 13432 ts_tablet_manager.cc:1905] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:17.434634 13430 tablet_replica.cc:330] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: stopping tablet replica
I20260807 08:37:17.434739 13430 raft_consensus.cc:2229] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:17.434863 13430 raft_consensus.cc:2258] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:17.434769 12642 catalog_manager.cc:4841] TS 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131): tablet 6aa3b558489345b8970df95816817768 (table test-table [id=ba2715885e4b41e48d875a1205a05f08]) successfully deleted
I20260807 08:37:17.435159 12857 tablet_service.cc:1467] Processing CreateTablet for tablet 4e864d52724d499a820a2dd2d25c7e1d (DEFAULT_TABLE table=test-table [id=d0c3c78312174fba802a88dab1160418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:17.435281 13397 tablet_bootstrap.cc:492] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Bootstrap starting.
I20260807 08:37:17.435317 12857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e864d52724d499a820a2dd2d25c7e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.435734 13397 tablet_bootstrap.cc:654] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.436270 13432 ts_tablet_manager.cc:1918] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:17.436343 13432 log.cc:1196] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/6aa3b558489345b8970df95816817768
I20260807 08:37:17.436372 13397 tablet_bootstrap.cc:492] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: No bootstrap required, opened a new log
I20260807 08:37:17.436410 13397 ts_tablet_manager.cc:1397] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:17.436529 13397 raft_consensus.cc:348] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.436589 13397 raft_consensus.cc:374] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.436602 13397 raft_consensus.cc:729] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab64f59afb54d1d99f4c06068b60670, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.436662 13432 ts_tablet_manager.cc:1939] T 6aa3b558489345b8970df95816817768 P 3ab64f59afb54d1d99f4c06068b60670: Deleting consensus metadata
I20260807 08:37:17.436672 13397 consensus_queue.cc:260] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.436810 13397 ts_tablet_manager.cc:1428] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 08:37:17.437016 13398 tablet_bootstrap.cc:492] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: Bootstrap starting.
I20260807 08:37:17.437108 12642 catalog_manager.cc:4841] TS 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3:39999): tablet 6aa3b558489345b8970df95816817768 (table test-table [id=ba2715885e4b41e48d875a1205a05f08]) successfully deleted
I20260807 08:37:17.437382 13430 ts_tablet_manager.cc:1905] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:17.437489 13398 tablet_bootstrap.cc:654] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.438731 13398 tablet_bootstrap.cc:492] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: No bootstrap required, opened a new log
I20260807 08:37:17.438804 13398 ts_tablet_manager.cc:1397] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:17.438980 13398 raft_consensus.cc:348] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.439097 13398 raft_consensus.cc:374] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.439131 13398 raft_consensus.cc:729] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d74056d0df64cf481a786defeb6621d, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.439193 13398 consensus_queue.cc:260] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.439324 13398 ts_tablet_manager.cc:1428] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:17.439200 13430 ts_tablet_manager.cc:1918] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:17.439575 13430 log.cc:1196] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/6aa3b558489345b8970df95816817768
I20260807 08:37:17.439879 13430 ts_tablet_manager.cc:1939] T 6aa3b558489345b8970df95816817768 P 7d74056d0df64cf481a786defeb6621d: Deleting consensus metadata
I20260807 08:37:17.440320 12643 catalog_manager.cc:4841] TS 7d74056d0df64cf481a786defeb6621d (127.10.207.1:41975): tablet 6aa3b558489345b8970df95816817768 (table test-table [id=ba2715885e4b41e48d875a1205a05f08]) successfully deleted
I20260807 08:37:17.447088 13399 tablet_bootstrap.cc:492] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: No bootstrap required, opened a new log
I20260807 08:37:17.447204 13399 ts_tablet_manager.cc:1397] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20260807 08:37:17.447435 13399 raft_consensus.cc:348] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.447523 13399 raft_consensus.cc:374] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.447554 13399 raft_consensus.cc:729] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3090408d76ad46159cc8aad858dbe5dd, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.447633 13399 consensus_queue.cc:260] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.447745 13399 ts_tablet_manager.cc:1428] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:17.537308 13404 raft_consensus.cc:482] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:17.537444 13404 raft_consensus.cc:504] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.537921 13404 leader_election.cc:288] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d74056d0df64cf481a786defeb6621d (127.10.207.1:41975), 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131)
W20260807 08:37:17.538657 13365 tablet.cc:2369] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:17.541206 13204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.538094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41975 (local address 127.10.207.3:46269)
0807 08:37:17.538170 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.538181 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.538224 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.538450 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.538452 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.538453 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:17.538563 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.538565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.539469 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.539470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.540266 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.540268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.540868 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.540868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.540993 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.540998 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.541061 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.541114 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 08:37:17.541244 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.538206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:46269 (local address 127.10.207.1:41975)
0807 08:37:17.538311 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.538314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.538322 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:17.538350 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.538351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.538361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.538404 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:17.538637 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.539349 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.540382 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.540772 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.540869 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.540882 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.541099 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.541162 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 08:37:17.541595 12889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e864d52724d499a820a2dd2d25c7e1d" candidate_uuid: "3ab64f59afb54d1d99f4c06068b60670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d74056d0df64cf481a786defeb6621d" is_pre_election: true
I20260807 08:37:17.541736 12889 raft_consensus.cc:2454] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab64f59afb54d1d99f4c06068b60670 in term 0.
I20260807 08:37:17.541971 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.538342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:57923 (local address 127.10.207.2:43131)
0807 08:37:17.538395 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.538397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.538406 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:17.538457 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.538458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.538473 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.538545 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:17.538828 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.539391 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.541000 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.541548 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.541628 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.541635 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.541872 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.541917 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:37:17.541957 13197 leader_election.cc:302] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab64f59afb54d1d99f4c06068b60670, 7d74056d0df64cf481a786defeb6621d; no voters:
I20260807 08:37:17.542099 13404 raft_consensus.cc:2781] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:17.542142 13404 raft_consensus.cc:482] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:17.542163 13404 raft_consensus.cc:3037] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.542315 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.538125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43131 (local address 127.10.207.3:57923)
0807 08:37:17.538290 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.538305 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.538373 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.538613 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.538614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.538616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:17.538723 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.538725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.539528 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.539529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.540829 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.540832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.541669 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.541669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.541780 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.541785 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.542147 (+ 362us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.542203 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260807 08:37:17.542586 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e864d52724d499a820a2dd2d25c7e1d" candidate_uuid: "3ab64f59afb54d1d99f4c06068b60670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3090408d76ad46159cc8aad858dbe5dd" is_pre_election: true
I20260807 08:37:17.542713 13089 raft_consensus.cc:2454] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab64f59afb54d1d99f4c06068b60670 in term 0.
I20260807 08:37:17.543130 13404 raft_consensus.cc:504] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.543303 13404 leader_election.cc:288] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [CANDIDATE]: Term 1 election: Requested vote from peers 7d74056d0df64cf481a786defeb6621d (127.10.207.1:41975), 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131)
I20260807 08:37:17.543491 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e864d52724d499a820a2dd2d25c7e1d" candidate_uuid: "3ab64f59afb54d1d99f4c06068b60670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3090408d76ad46159cc8aad858dbe5dd"
I20260807 08:37:17.543502 12889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e864d52724d499a820a2dd2d25c7e1d" candidate_uuid: "3ab64f59afb54d1d99f4c06068b60670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d74056d0df64cf481a786defeb6621d"
I20260807 08:37:17.543587 12889 raft_consensus.cc:3037] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.543591 13089 raft_consensus.cc:3037] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.544368 13089 raft_consensus.cc:2454] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab64f59afb54d1d99f4c06068b60670 in term 1.
I20260807 08:37:17.544531 12889 raft_consensus.cc:2454] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab64f59afb54d1d99f4c06068b60670 in term 1.
I20260807 08:37:17.544620 13196 leader_election.cc:302] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3090408d76ad46159cc8aad858dbe5dd, 3ab64f59afb54d1d99f4c06068b60670; no voters:
I20260807 08:37:17.544723 13404 raft_consensus.cc:2781] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:17.544898 13404 raft_consensus.cc:686] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 1 LEADER]: Becoming Leader. State: Replica: 3ab64f59afb54d1d99f4c06068b60670, State: Running, Role: LEADER
I20260807 08:37:17.544976 13404 consensus_queue.cc:237] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.545827 12664 catalog_manager.cc:5495] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3). New cstate: current_term: 1 leader_uuid: "3ab64f59afb54d1d99f4c06068b60670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } health_report { overall_health: HEALTHY } } }
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091836, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:17.591508 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.583608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.10.207.62:41295)
0807 08:37:17.583719 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.583723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.583865 (+ 142us) server_negotiation.cc:407] Connection header received
0807 08:37:17.583981 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.583983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.583994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.584071 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.584671 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.585206 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.585923 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.586118 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.586196 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.588735 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.588737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.588737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.588744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.590081 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.590503 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.590504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.590505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.590636 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.590841 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.590842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.590843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.590961 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.591044 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.591268 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.591357 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:37:17.591512 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.583499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41295 (local address 127.0.0.1:34964)
0807 08:37:17.583772 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.583784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.583893 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.584148 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.584149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.584508 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:17.584569 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.584572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.585292 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.585294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.585835 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.585836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.586207 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.586208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.586269 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.586916 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:17.586921 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:17.588620 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:17.590195 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.590203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.590206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.590410 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.590709 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:17.590710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:17.590710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:17.590776 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:17.591050 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:17.591058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:17.591159 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.591243 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.591416 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":71,"thread_start_us":171,"threads_started":1}
I20260807 08:37:17.594452 12664 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:34964:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:37:17.594949 12664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:17.597090 12857 tablet_service.cc:1467] Processing CreateTablet for tablet a6bb55ad35954db9bc6317479ba99e8c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5eddaf30dc1c46468ad4e182b51fbc6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:17.597128 13274 tablet_service.cc:1467] Processing CreateTablet for tablet a6bb55ad35954db9bc6317479ba99e8c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5eddaf30dc1c46468ad4e182b51fbc6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:17.597303 13274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bb55ad35954db9bc6317479ba99e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.597306 12857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bb55ad35954db9bc6317479ba99e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.597618 13060 tablet_service.cc:1467] Processing CreateTablet for tablet a6bb55ad35954db9bc6317479ba99e8c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5eddaf30dc1c46468ad4e182b51fbc6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:17.597851 13060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bb55ad35954db9bc6317479ba99e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:17.599277 13398 tablet_bootstrap.cc:492] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: Bootstrap starting.
I20260807 08:37:17.599570 13399 tablet_bootstrap.cc:492] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: Bootstrap starting.
I20260807 08:37:17.599696 13398 tablet_bootstrap.cc:654] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.600104 13399 tablet_bootstrap.cc:654] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.600435 13398 tablet_bootstrap.cc:492] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: No bootstrap required, opened a new log
I20260807 08:37:17.600487 13398 ts_tablet_manager.cc:1397] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:17.600620 13398 raft_consensus.cc:348] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.600710 13398 raft_consensus.cc:374] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.600730 13398 raft_consensus.cc:729] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d74056d0df64cf481a786defeb6621d, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.600801 13398 consensus_queue.cc:260] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.600898 13399 tablet_bootstrap.cc:492] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: No bootstrap required, opened a new log
I20260807 08:37:17.600904 13398 ts_tablet_manager.cc:1428] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:17.600955 13399 ts_tablet_manager.cc:1397] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:17.601032 13412 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.601111 13412 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:17.601199 13399 raft_consensus.cc:348] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
W20260807 08:37:17.601264 13158 tablet.cc:2369] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:17.601320 13399 raft_consensus.cc:374] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.601347 13399 raft_consensus.cc:729] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3090408d76ad46159cc8aad858dbe5dd, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.601415 13399 consensus_queue.cc:260] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.601521 13399 ts_tablet_manager.cc:1428] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:17.601698 13407 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.601760 13397 tablet_bootstrap.cc:492] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: Bootstrap starting.
I20260807 08:37:17.601804 13407 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:17.602278 13397 tablet_bootstrap.cc:654] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:17.603062 13397 tablet_bootstrap.cc:492] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: No bootstrap required, opened a new log
I20260807 08:37:17.603116 13397 ts_tablet_manager.cc:1397] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:17.603282 13397 raft_consensus.cc:348] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.603396 13397 raft_consensus.cc:374] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:17.603425 13397 raft_consensus.cc:729] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab64f59afb54d1d99f4c06068b60670, State: Initialized, Role: FOLLOWER
I20260807 08:37:17.603475 13397 consensus_queue.cc:260] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.603585 13397 ts_tablet_manager.cc:1428] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:17.603634 13404 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.604261 13404 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:17.604916 13412 raft_consensus.cc:482] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:17.604987 13412 raft_consensus.cc:504] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.605114 13412 leader_election.cc:288] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131), 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3:39999)
I20260807 08:37:17.605293 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb55ad35954db9bc6317479ba99e8c" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3090408d76ad46159cc8aad858dbe5dd" is_pre_election: true
I20260807 08:37:17.605335 13300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb55ad35954db9bc6317479ba99e8c" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab64f59afb54d1d99f4c06068b60670" is_pre_election: true
I20260807 08:37:17.605400 13300 raft_consensus.cc:2454] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 0.
I20260807 08:37:17.605408 13089 raft_consensus.cc:2454] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 0.
I20260807 08:37:17.605543 12786 leader_election.cc:302] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab64f59afb54d1d99f4c06068b60670, 7d74056d0df64cf481a786defeb6621d; no voters:
I20260807 08:37:17.605657 13412 raft_consensus.cc:2781] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:17.605721 13412 raft_consensus.cc:482] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:17.605762 13412 raft_consensus.cc:3037] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.606484 13412 raft_consensus.cc:504] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.606701 13412 leader_election.cc:288] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 election: Requested vote from peers 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131), 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3:39999)
I20260807 08:37:17.606899 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb55ad35954db9bc6317479ba99e8c" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3090408d76ad46159cc8aad858dbe5dd"
I20260807 08:37:17.606920 13300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb55ad35954db9bc6317479ba99e8c" candidate_uuid: "7d74056d0df64cf481a786defeb6621d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab64f59afb54d1d99f4c06068b60670"
I20260807 08:37:17.606990 13300 raft_consensus.cc:3037] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.607023 13089 raft_consensus.cc:3037] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:17.607728 13089 raft_consensus.cc:2454] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 1.
I20260807 08:37:17.607894 13300 raft_consensus.cc:2454] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d74056d0df64cf481a786defeb6621d in term 1.
I20260807 08:37:17.607957 12786 leader_election.cc:302] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3090408d76ad46159cc8aad858dbe5dd, 7d74056d0df64cf481a786defeb6621d; no voters:
I20260807 08:37:17.608078 13412 raft_consensus.cc:2781] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:17.608150 13412 raft_consensus.cc:686] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [term 1 LEADER]: Becoming Leader. State: Replica: 7d74056d0df64cf481a786defeb6621d, State: Running, Role: LEADER
I20260807 08:37:17.608209 13412 consensus_queue.cc:237] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } }
I20260807 08:37:17.608536 13405 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: TxnStatusTablet state changed. Reason: New leader 7d74056d0df64cf481a786defeb6621d. Latest consensus state: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.608642 13405 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:17.608815 13454 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:37:17.608840 12661 catalog_manager.cc:5495] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d74056d0df64cf481a786defeb6621d (127.10.207.1). New cstate: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:17.608978 13454 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:37:17.616694 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.613863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.10.207.1:41975)
0807 08:37:17.613941 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.613943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.613954 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:17.613979 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.613980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.613990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.614068 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:17.614265 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.615017 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.615846 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.616148 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.616228 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.616437 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.616473 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.616535 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.616603 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.616625 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:37:17.616719 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.613614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41975 (local address 127.0.0.1:34376)
0807 08:37:17.613720 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.613733 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.613779 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.614122 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.614123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.614125 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:17.614183 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.614186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.615130 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.615131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.615750 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.615752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.616254 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.616255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.616314 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.616331 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.616548 (+ 217us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.616556 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.616596 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.616654 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 08:37:17.618017 13089 raft_consensus.cc:1261] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Refusing update from remote peer 7d74056d0df64cf481a786defeb6621d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:17.618036 13300 raft_consensus.cc:1261] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Refusing update from remote peer 7d74056d0df64cf481a786defeb6621d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:17.618283 13412 consensus_queue.cc:1035] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:17.618402 13412 consensus_queue.cc:1035] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:17.618919 13407 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: TxnStatusTablet state changed. Reason: New leader 7d74056d0df64cf481a786defeb6621d. Latest consensus state: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.619026 13407 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:17.619091 13404 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: TxnStatusTablet state changed. Reason: New leader 7d74056d0df64cf481a786defeb6621d. Latest consensus state: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.619196 13404 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:17.620302 13412 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.620409 13412 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:17.620914 13407 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.621021 13407 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 3090408d76ad46159cc8aad858dbe5dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:17.623303 13412 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.623405 13412 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 7d74056d0df64cf481a786defeb6621d: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:17.623492 13418 mvcc.cc:205] Tried to move back new op lower bound from 7315832166881677312 to 7315832166843600896. Current Snapshot: MvccSnapshot[applied={T|T < 7315832166881677312}]
I20260807 08:37:17.623613 13404 tablet_replica.cc:439] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d74056d0df64cf481a786defeb6621d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 } } peers { permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 } } peers { permanent_uuid: "3ab64f59afb54d1d99f4c06068b60670" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39999 } } }
I20260807 08:37:17.623670 13404 tablet_replica.cc:442] T a6bb55ad35954db9bc6317479ba99e8c P 3ab64f59afb54d1d99f4c06068b60670: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:17.626169 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.622814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54894 (local address 127.10.207.3:39999)
0807 08:37:17.622862 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.622864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.622875 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:17.622951 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.622952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.622977 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.623058 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:17.623552 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.624231 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.625311 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.625496 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.625581 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.625756 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.625876 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.625938 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.626033 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.626053 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:37:17.631862 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.629168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:54685 (local address 127.10.207.1:41975)
0807 08:37:17.629221 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.629223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.629234 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:17.629293 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.629294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.629306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.629368 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:17.629511 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.630194 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.631049 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.631340 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.631415 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.631602 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.631635 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.631695 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.631776 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.631795 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:37:17.631954 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.629076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41975 (local address 127.10.207.3:54685)
0807 08:37:17.629176 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.629191 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.629235 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.629381 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.629382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.629383 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:17.629437 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.629438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.630281 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.630282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.630948 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.630950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.631427 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.631428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.631486 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.631501 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.631709 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.631711 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.631769 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.631839 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 08:37:17.634735 13089 raft_consensus.cc:1261] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Refusing update from remote peer 3ab64f59afb54d1d99f4c06068b60670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:17.634824 12889 raft_consensus.cc:1261] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Refusing update from remote peer 3ab64f59afb54d1d99f4c06068b60670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:17.635010 13404 consensus_queue.cc:1035] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3090408d76ad46159cc8aad858dbe5dd" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:17.635162 13440 consensus_queue.cc:1035] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d74056d0df64cf481a786defeb6621d" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:17.646186 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.642891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:39999 (local address 127.10.207.1:45171)
0807 08:37:17.643097 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:17.643108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:17.643168 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:17.643328 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:17.643328 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:17.643330 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:17.643380 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.643381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.644260 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.644261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.645205 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:17.645208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.645636 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.645637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:17.645697 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.645714 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.645989 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.645991 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:17.646055 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:37:17.646102 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":87,"threads_started":1}
I20260807 08:37:17.646188 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.642961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:45171 (local address 127.10.207.3:39999)
0807 08:37:17.643007 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.643009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.643216 (+ 207us) server_negotiation.cc:407] Connection header received
0807 08:37:17.643244 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.643245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.643256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.643324 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:17.643426 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.644137 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.645319 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.645543 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.645612 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.645830 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:17.645866 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:17.645973 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.646060 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.646097 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:37:17.655534 12662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:17.655659 12662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:17.656292 12662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa8a1f7704f4ee6b32a08936b1aa172: Sending DeleteTablet for 3 replicas of tablet 4e864d52724d499a820a2dd2d25c7e1d
I20260807 08:37:17.656641 13060 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e864d52724d499a820a2dd2d25c7e1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:17 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37728
I20260807 08:37:17.656692 13274 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e864d52724d499a820a2dd2d25c7e1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:17 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:54888
I20260807 08:37:17.656840 13431 tablet_replica.cc:330] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: stopping tablet replica
I20260807 08:37:17.656906 13431 raft_consensus.cc:2229] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:17.656949 13431 raft_consensus.cc:2258] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:17.657029 13432 tablet_replica.cc:330] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: stopping tablet replica
I20260807 08:37:17.657078 13432 raft_consensus.cc:2229] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:17.657177 13432 raft_consensus.cc:2258] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:17.657271 12857 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e864d52724d499a820a2dd2d25c7e1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:17 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:34350
I20260807 08:37:17.657399 13430 tablet_replica.cc:330] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: stopping tablet replica
I20260807 08:37:17.657488 13430 raft_consensus.cc:2229] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:17.657529 13430 raft_consensus.cc:2258] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:17.657922 13432 ts_tablet_manager.cc:1905] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:17.657938 13430 ts_tablet_manager.cc:1905] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:17.659862 13432 ts_tablet_manager.cc:1918] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:17.659938 13432 log.cc:1196] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/4e864d52724d499a820a2dd2d25c7e1d
I20260807 08:37:17.659929 13430 ts_tablet_manager.cc:1918] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:17.659978 13430 log.cc:1196] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/4e864d52724d499a820a2dd2d25c7e1d
I20260807 08:37:17.660190 13432 ts_tablet_manager.cc:1939] T 4e864d52724d499a820a2dd2d25c7e1d P 3ab64f59afb54d1d99f4c06068b60670: Deleting consensus metadata
I20260807 08:37:17.660212 13430 ts_tablet_manager.cc:1939] T 4e864d52724d499a820a2dd2d25c7e1d P 7d74056d0df64cf481a786defeb6621d: Deleting consensus metadata
I20260807 08:37:17.660642 12642 catalog_manager.cc:4841] TS 3ab64f59afb54d1d99f4c06068b60670 (127.10.207.3:39999): tablet 4e864d52724d499a820a2dd2d25c7e1d (table test-table [id=d0c3c78312174fba802a88dab1160418]) successfully deleted
I20260807 08:37:17.660732 12643 catalog_manager.cc:4841] TS 7d74056d0df64cf481a786defeb6621d (127.10.207.1:41975): tablet 4e864d52724d499a820a2dd2d25c7e1d (table test-table [id=d0c3c78312174fba802a88dab1160418]) successfully deleted
I20260807 08:37:17.661659 13431 ts_tablet_manager.cc:1905] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:17.663549 13431 ts_tablet_manager.cc:1918] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:17.663638 13431 log.cc:1196] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/4e864d52724d499a820a2dd2d25c7e1d
I20260807 08:37:17.663883 13431 ts_tablet_manager.cc:1939] T 4e864d52724d499a820a2dd2d25c7e1d P 3090408d76ad46159cc8aad858dbe5dd: Deleting consensus metadata
I20260807 08:37:17.664288 12642 catalog_manager.cc:4841] TS 3090408d76ad46159cc8aad858dbe5dd (127.10.207.2:43131): tablet 4e864d52724d499a820a2dd2d25c7e1d (table test-table [id=d0c3c78312174fba802a88dab1160418]) successfully deleted
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[12600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:17.673453 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.661570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.10.207.1:41975)
0807 08:37:17.661612 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.661614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.662698 (+ 1084us) server_negotiation.cc:407] Connection header received
0807 08:37:17.662800 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.662801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.662812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.662842 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.663694 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.664428 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.665550 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.665837 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.665876 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.669955 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.669967 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.669968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.669975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.672101 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.672493 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.672496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.672497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.672558 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.672755 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.672755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.672756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.672911 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.673036 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.673250 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.673333 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 08:37:17.674332 12944 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34388
I20260807 08:37:17.684294 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:17.675644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.10.207.62:41295)
0807 08:37:17.675695 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:17.675697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:17.675963 (+ 266us) server_negotiation.cc:407] Connection header received
0807 08:37:17.676010 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:17.676011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:17.676028 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:17.676101 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:17.676779 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.677629 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.678769 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:17.679118 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:17.679195 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:17.680650 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:17.680651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:17.680652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:17.680660 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:17.682670 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.683098 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.683100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.683100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.683176 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:17.683433 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:17.683434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:17.683434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:17.683602 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:17.683720 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:17.684083 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 08:37:17.684180 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 08:37:17.684558 12662 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34972
I20260807 08:37:17.685727 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12759
I20260807 08:37:17.703212 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12966
I20260807 08:37:17.714380 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 13172
I20260807 08:37:17.723148 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12618
2026-08-07T08:37:17Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2598 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[13488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[13488](info): set up 2 sockets
Aug 07 08:37:17 dist-test-slave-h6n0 krb5kdc[13488](info): commencing operation
krb5kdc: starting...
W20260807 08:37:19.001137 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:19.142714 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43719
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:19.243162 13511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:19.243468 13511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:19.243515 13511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:19.244917 13511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:19.244997 13511 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:37:19.245029 13511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:19.245067 13511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:19.245097 13511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:19.245162 13511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:19.247092 13511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43719
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:19.247478 13511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:19.247792 13511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:19.253985 13520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.253999 13518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.253991 13517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:19.254474 13511 server_base.cc:1034] running on GCE node
I20260807 08:37:19.254784 13511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:19.255085 13511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:19.256265 13511 hybrid_clock.cc:648] HybridClock initialized: now 1786091839256248 us; error 40 us; skew 500 ppm
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:19.259392 13511 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:19.260155 13511 webserver.cc:460] Webserver started at http://127.10.207.62:45383/ using document root <none> and password file <none>
I20260807 08:37:19.260514 13511 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:19.260600 13511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:19.260767 13511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:19.261979 13511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "07a64e43f5b243d3a5c876c2b0ec9177"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "6e4b657c4aff3a90d5f6357d786a188f"
server_key_iv: "8fe9a7be6f7107903fd19a9b4ed091da"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.262408 13511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "07a64e43f5b243d3a5c876c2b0ec9177"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "6e4b657c4aff3a90d5f6357d786a188f"
server_key_iv: "8fe9a7be6f7107903fd19a9b4ed091da"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.264045 13511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:19.265009 13531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.265316 13511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:19.265429 13511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "07a64e43f5b243d3a5c876c2b0ec9177"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "6e4b657c4aff3a90d5f6357d786a188f"
server_key_iv: "8fe9a7be6f7107903fd19a9b4ed091da"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.265522 13511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:19.294875 13511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:19.297020 13511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:19.297199 13511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:19.302637 13511 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43719
I20260807 08:37:19.302739 13622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43719 every 8 connection(s)
I20260807 08:37:19.303040 13511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:19.303887 13623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:19.306833 13623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Bootstrap starting.
I20260807 08:37:19.307596 13623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:19.307955 13623 log.cc:824] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:19.308873 13623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: No bootstrap required, opened a new log
I20260807 08:37:19.310915 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 13511
I20260807 08:37:19.311043 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:19.311267 11068 external_mini_cluster.cc:1428] Setting key 44614f5660d510baffdc1f57524032a5
I20260807 08:37:19.311605 13623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43719 } }
I20260807 08:37:19.311800 13623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:19.311838 13623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a64e43f5b243d3a5c876c2b0ec9177, State: Initialized, Role: FOLLOWER
I20260807 08:37:19.312021 13623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43719 } }
I20260807 08:37:19.312151 13623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:19.312201 13623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:19.312276 13623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:19.312947 13623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43719 } }
I20260807 08:37:19.313088 13623 leader_election.cc:302] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07a64e43f5b243d3a5c876c2b0ec9177; no voters:
I20260807 08:37:19.313326 13623 leader_election.cc:288] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:19.313354 13630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:19.313560 13630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [term 1 LEADER]: Becoming Leader. State: Replica: 07a64e43f5b243d3a5c876c2b0ec9177, State: Running, Role: LEADER
I20260807 08:37:19.313650 13623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:19.313657 13630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43719 } }
I20260807 08:37:19.314108 13630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07a64e43f5b243d3a5c876c2b0ec9177. Latest consensus state: current_term: 1 leader_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43719 } } }
I20260807 08:37:19.314213 13630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:19.314402 13633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a64e43f5b243d3a5c876c2b0ec9177" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43719 } } }
I20260807 08:37:19.314486 13633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:19.314643 13638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:19.315323 13638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:19.318109 13638 catalog_manager.cc:1347] Generated new cluster ID: b4520371fc3b447796f50abde67eb7d6
I20260807 08:37:19.318162 13638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:19.323752 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.311888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.10.207.62:43719)
0807 08:37:19.312146 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.312155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.312210 (+ 55us) server_negotiation.cc:407] Connection header received
0807 08:37:19.312386 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.312394 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.312468 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.312627 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.313485 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.314946 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.316040 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.316569 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.316696 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.319863 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.319867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.319867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.319878 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.321891 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.322406 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.322409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.322409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.322551 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.322800 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.322801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.322801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.323085 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.323263 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.323405 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.323546 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":121,"threads_started":1}
I20260807 08:37:19.332551 13638 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:19.333063 13638 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:19.338461 13638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Generated new TSK 0
I20260807 08:37:19.338768 13638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:19.372742 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43719
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:19.464762 13660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:19.464991 13660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:19.465030 13660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:19.466136 13660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:19.466207 13660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:19.466241 13660 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:37:19.466274 13660 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:19.466303 13660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:19.469107 13660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:43719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:19.469393 13660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:19.469683 13660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:19.476382 13668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.476394 13669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.476517 13673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:19.476682 13660 server_base.cc:1034] running on GCE node
I20260807 08:37:19.476895 13660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:19.477147 13660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:19.478351 13660 hybrid_clock.cc:648] HybridClock initialized: now 1786091839478314 us; error 54 us; skew 500 ppm
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:19.481175 13660 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:19.481709 13660 webserver.cc:460] Webserver started at http://127.10.207.1:34775/ using document root <none> and password file <none>
I20260807 08:37:19.481953 13660 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:19.482029 13660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:19.482168 13660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:19.483381 13660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "fe2b305201c24ef2be1df3192dad1624"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "4db139a52cf2554aa140716ea414165e"
server_key_iv: "deb247638e0484a9e4f594718dd49b9b"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.483779 13660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "fe2b305201c24ef2be1df3192dad1624"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "4db139a52cf2554aa140716ea414165e"
server_key_iv: "deb247638e0484a9e4f594718dd49b9b"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.485212 13660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:19.486129 13682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.486373 13660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:19.486486 13660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "fe2b305201c24ef2be1df3192dad1624"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "4db139a52cf2554aa140716ea414165e"
server_key_iv: "deb247638e0484a9e4f594718dd49b9b"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.486634 13660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:19.496693 13660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:19.498703 13660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:19.498880 13660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:19.499516 13660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:19.499572 13660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.499605 13660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:19.499626 13660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:19.508033 13660 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:34629
I20260807 08:37:19.508230 13853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:34629 every 8 connection(s)
I20260807 08:37:19.508504 13660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:19.511294 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.500130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:48729 (local address 127.10.207.62:43719)
0807 08:37:19.500234 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.500237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.500543 (+ 306us) server_negotiation.cc:407] Connection header received
0807 08:37:19.501159 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.501161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.501182 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.501237 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.501964 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.502830 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.504050 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.504300 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.504387 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.507899 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.507901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.507902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.507913 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.509726 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.510185 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.510187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.510187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.510257 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.510554 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.510555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.510555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.510691 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.510772 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.511078 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.511189 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:37:19.511490 13693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.500081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43719 (local address 127.10.207.1:48729)
0807 08:37:19.500368 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.500418 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.501009 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.501324 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.501328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.501628 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:19.501852 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.501856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.502938 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.502939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.503944 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.503947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.504396 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.504397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.504536 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.505247 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:19.505259 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:19.507753 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:19.509835 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.509842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.509845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.510100 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.510338 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.510339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.510340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.510462 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.510775 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:19.510784 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:19.510934 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.511054 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.511240 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":155,"threads_started":1}
I20260807 08:37:19.512167 13856 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43719
I20260807 08:37:19.512267 13856 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:19.512487 13856 heartbeater.cc:511] Master 127.10.207.62:43719 requested a full tablet report, sending...
I20260807 08:37:19.513361 13552 ts_manager.cc:194] Registered new tserver with Master: fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629)
I20260807 08:37:19.513893 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 13660
I20260807 08:37:19.514007 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:19.514269 11068 external_mini_cluster.cc:1428] Setting key 679b138f06d87f608b6a5b448e3e3c74
I20260807 08:37:19.514387 13552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:48729
I20260807 08:37:19.520539 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.513236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43719 (local address 127.10.207.1:57859)
0807 08:37:19.513432 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.513443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.513546 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.513752 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.513754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.514031 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:19.514133 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.514136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.515159 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.515160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.516121 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.516124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.516515 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.516517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.516602 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.517144 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:19.517150 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:19.517455 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:19.519532 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.519532 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.519533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.519699 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.519936 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.519936 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.519937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.519991 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.520295 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:19.520296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:19.520332 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.520388 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.520465 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260807 08:37:19.520696 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.513308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:57859 (local address 127.10.207.62:43719)
0807 08:37:19.513353 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.513355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.513519 (+ 164us) server_negotiation.cc:407] Connection header received
0807 08:37:19.513629 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.513630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.513643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.513705 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.514237 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.515039 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.516222 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.516415 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.516481 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.517557 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.517559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.517559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.517567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.519428 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.519787 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.519788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.519789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.519859 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.520066 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.520067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.520068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.520222 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.520297 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.520439 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.520533 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:19.545439 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43719
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:19.637483 13875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:19.637719 13875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:19.637744 13875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:19.638834 13875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:19.638873 13875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:19.638893 13875 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:37:19.638906 13875 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:19.638922 13875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:19.640367 13875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:43719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:19.640583 13875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:19.640786 13875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:19.646459 13883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.646463 13882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.646718 13886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:19.646867 13875 server_base.cc:1034] running on GCE node
I20260807 08:37:19.647109 13875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:19.647385 13875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:19.648557 13875 hybrid_clock.cc:648] HybridClock initialized: now 1786091839648547 us; error 29 us; skew 500 ppm
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:19.651474 13875 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:19.652107 13875 webserver.cc:460] Webserver started at http://127.10.207.2:43497/ using document root <none> and password file <none>
I20260807 08:37:19.652398 13875 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:19.652473 13875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:19.652633 13875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:19.653894 13875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "2ba3526d1b7c486184a08448c20ae18e"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "71ab6db1cc7d51223aebc320d582f04b"
server_key_iv: "d203c19fbba4033a19297b5cd8a2e7d1"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.654340 13875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "2ba3526d1b7c486184a08448c20ae18e"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "71ab6db1cc7d51223aebc320d582f04b"
server_key_iv: "d203c19fbba4033a19297b5cd8a2e7d1"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.655951 13875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:19.656944 13900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.657194 13875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:19.657317 13875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "2ba3526d1b7c486184a08448c20ae18e"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "71ab6db1cc7d51223aebc320d582f04b"
server_key_iv: "d203c19fbba4033a19297b5cd8a2e7d1"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.657398 13875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:19.688194 13875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:19.690872 13875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:19.691061 13875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:19.691771 13875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:19.691838 13875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.691888 13875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:19.691911 13875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:19.701061 13875 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:35659
I20260807 08:37:19.701180 14062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:35659 every 8 connection(s)
I20260807 08:37:19.701560 13875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:19.701937 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 13875
I20260807 08:37:19.702049 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:19.702279 11068 external_mini_cluster.cc:1428] Setting key 5b81479be6577b0810c1e90affa8da61
I20260807 08:37:19.703264 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.692548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:53191 (local address 127.10.207.62:43719)
0807 08:37:19.692615 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.692617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.692934 (+ 317us) server_negotiation.cc:407] Connection header received
0807 08:37:19.693669 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.693672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.693692 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.693782 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.694800 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.695403 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.696468 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.696666 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.696761 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.699516 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.699518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.699519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.699527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.701410 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.701974 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.701975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.701976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.702055 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.702356 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.702356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.702357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.702497 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.702600 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.702989 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.703152 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:37:19.703312 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.692548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43719 (local address 127.10.207.2:53191)
0807 08:37:19.692789 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.692831 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.693443 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.693892 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.693897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.694344 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:19.694687 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.694693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.695509 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.695510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.696360 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.696362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.696774 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.696775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.696922 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.697475 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:19.697487 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:19.699404 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:19.701517 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.701525 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.701527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.701876 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.702130 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.702131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.702131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.702270 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.702634 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:19.702642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:19.702801 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.702895 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.703075 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":134,"threads_started":1}
I20260807 08:37:19.703935 14063 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43719
I20260807 08:37:19.704051 14063 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:19.704279 14063 heartbeater.cc:511] Master 127.10.207.62:43719 requested a full tablet report, sending...
I20260807 08:37:19.704715 13552 ts_manager.cc:194] Registered new tserver with Master: 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659)
I20260807 08:37:19.705231 13552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:53191
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
I20260807 08:37:19.712898 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.705257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:60653 (local address 127.10.207.62:43719)
0807 08:37:19.705412 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.705414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.706763 (+ 1349us) server_negotiation.cc:407] Connection header received
0807 08:37:19.706843 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.706844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.706856 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.706910 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.707270 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.707842 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.708615 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.708828 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.708916 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.709980 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.709982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.709982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.709989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.711602 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.712010 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.712011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.712012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.712082 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.712296 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.712297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.712297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.712472 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.712556 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.712723 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.712799 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260807 08:37:19.712904 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.705236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43719 (local address 127.10.207.2:60653)
0807 08:37:19.706676 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.706690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.706780 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.706917 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.706917 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.707128 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:19.707186 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.707188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.707946 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.707947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.708500 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.708502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.708919 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.708920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.709002 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.709546 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:19.709554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:19.709866 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:19.711736 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.711737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.711737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.711912 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.712156 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.712157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.712157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.712225 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.712560 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:19.712562 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:19.712620 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.712686 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.712803 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1408,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:19.730484 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43719
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:19.819339 14077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:19.819550 14077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:19.819577 14077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:19.820631 14077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:19.820676 14077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:19.820699 14077 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:37:19.820713 14077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:19.820732 14077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:19.822120 14077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:43719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:19.822328 14077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:19.822506 14077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:19.827894 14085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.827976 14087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:19.827981 14084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:19.828197 14077 server_base.cc:1034] running on GCE node
I20260807 08:37:19.828574 14077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:19.828928 14077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:19.830142 14077 hybrid_clock.cc:648] HybridClock initialized: now 1786091839830133 us; error 49 us; skew 500 ppm
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:19.833084 14077 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:19.833654 14077 webserver.cc:460] Webserver started at http://127.10.207.3:39797/ using document root <none> and password file <none>
I20260807 08:37:19.833941 14077 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:19.834019 14077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:19.834201 14077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:19.835494 14077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "7153cd84417544629c89a9091820481a"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "3d1053f5010e93354158665489cdd1cd"
server_key_iv: "bded45cda0a6351d67471786a27535f0"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.836071 14077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "7153cd84417544629c89a9091820481a"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "3d1053f5010e93354158665489cdd1cd"
server_key_iv: "bded45cda0a6351d67471786a27535f0"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.837749 14077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:19.839071 14097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.839332 14077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:37:19.839434 14077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "7153cd84417544629c89a9091820481a"
format_stamp: "Formatted at 2026-08-07 08:37:19 on dist-test-slave-h6n0"
server_key: "3d1053f5010e93354158665489cdd1cd"
server_key_iv: "bded45cda0a6351d67471786a27535f0"
server_key_version: "encryptionkey@0"
I20260807 08:37:19.839524 14077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:19.859262 14077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:19.861582 14077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:19.861739 14077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:19.862267 14077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:19.862317 14077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.862340 14077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:19.862353 14077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:19.870770 14077 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:40539
I20260807 08:37:19.870884 14270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:40539 every 8 connection(s)
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:19.871193 14077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:19.875356 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.863168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:42775 (local address 127.10.207.62:43719)
0807 08:37:19.863262 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.863266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.863510 (+ 244us) server_negotiation.cc:407] Connection header received
0807 08:37:19.864405 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.864407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.864424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.864521 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.865572 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.866433 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.868160 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.868553 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.868638 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.871964 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.871966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.871967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.871977 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.873620 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.874123 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.874124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.874125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.874195 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.874483 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.874483 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.874484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.874694 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.874782 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.875147 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.875243 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:37:19.875583 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.863102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43719 (local address 127.10.207.3:42775)
0807 08:37:19.863340 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.863393 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.864253 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.864624 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.864629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.865053 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:19.865455 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.865461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.866595 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.866598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.868024 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.868029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.868663 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.868664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.868840 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.869558 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:19.869569 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:19.871761 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:19.873748 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.873756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.873759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.874026 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.874273 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.874274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.874274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.874401 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.874771 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:19.874778 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:19.874936 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.875121 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.875324 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":153,"threads_started":1}
I20260807 08:37:19.876205 14271 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43719
I20260807 08:37:19.876325 14271 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:19.876564 14271 heartbeater.cc:511] Master 127.10.207.62:43719 requested a full tablet report, sending...
I20260807 08:37:19.876649 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 14077
I20260807 08:37:19.876724 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:19.876935 11068 external_mini_cluster.cc:1428] Setting key 173a79df2b24b91f6b724c7ea3e7fbe7
I20260807 08:37:19.877094 13550 ts_manager.cc:194] Registered new tserver with Master: 7153cd84417544629c89a9091820481a (127.10.207.3:40539)
I20260807 08:37:19.877463 13550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:42775
I20260807 08:37:19.878260 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:19.884600 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.877852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:58329 (local address 127.10.207.62:43719)
0807 08:37:19.877898 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.877900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.878085 (+ 185us) server_negotiation.cc:407] Connection header received
0807 08:37:19.878240 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.878241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.878253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.878303 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.878726 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.879234 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.879958 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.880167 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.880260 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.881450 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.881452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.881452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.881459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.883135 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.883606 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.883608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.883608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.883670 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.883960 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.883961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.883961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.884110 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.884195 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.884448 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.884537 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:37:19.884654 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.877780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43719 (local address 127.10.207.3:58329)
0807 08:37:19.877997 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.878007 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.878172 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.878309 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.878310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.878511 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:19.878632 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.878640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.879349 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.879350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.879871 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.879872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.880263 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.880264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.880347 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.880850 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:19.880857 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:19.881317 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:19.883223 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.883224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.883225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.883474 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.883751 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:19.883752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:19.883753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:19.883835 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:19.884210 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:19.884212 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:19.884280 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.884376 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.884509 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:37:19 dist-test-slave-h6n0 krb5kdc[13488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:19.907073 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.899245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50466 (local address 127.10.207.62:43719)
0807 08:37:19.899314 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.899316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.899323 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:37:19.899341 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.899342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.899351 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.899422 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:19.899999 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.900484 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.901218 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.901412 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.901512 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.904277 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:19.904278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:19.904278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:19.904284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:19.905701 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.906135 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.906136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.906136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.906187 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:19.906477 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:19.906478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:19.906478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:19.906660 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:19.906754 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.906945 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.907012 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:37:19.909138 13550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:19.911470 13550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:19.919591 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.915177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35659 (local address 127.0.0.1:35810)
0807 08:37:19.915311 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.915324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.915370 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.916208 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.916209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.916212 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:19.916260 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.916263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.917483 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.917484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.918474 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.918477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.919218 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.919219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.919314 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.919327 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.919422 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.919486 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260807 08:37:19.919595 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.915714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35810 (local address 127.10.207.2:35659)
0807 08:37:19.915947 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.915952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.915963 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:19.916008 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.916011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.916028 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.916124 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:19.916373 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.917383 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.918611 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.919109 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.919203 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.919265 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.919443 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.919481 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":99,"threads_started":1}
I20260807 08:37:19.919800 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.914914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:34629 (local address 127.0.0.1:56350)
0807 08:37:19.915262 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.915277 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.915350 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.915951 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.915952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.915956 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:19.916040 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.916042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.917077 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.917078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.918487 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.918491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.919530 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.919531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.919617 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.919622 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.919656 (+ 34us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.919712 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":58,"spinlock_wait_cycles":768,"thread_start_us":150,"threads_started":1}
I20260807 08:37:19.920167 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.914898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:40539 (local address 127.0.0.1:33248)
0807 08:37:19.915061 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:19.915107 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:19.915172 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:19.915519 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:19.915523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:19.915525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:19.915599 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.915601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.916921 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.916922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.918490 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:19.918491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.919528 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.919529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:19.919982 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.919986 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:19.920046 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:37:19.920108 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":490,"thread_start_us":105,"threads_started":1}
I20260807 08:37:19.920271 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.915081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.10.207.3:40539)
0807 08:37:19.915308 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.915313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.915326 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:37:19.915377 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.915379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.915399 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.915490 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:19.915713 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.916815 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.918637 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.919415 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.919574 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.919648 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.920096 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.920148 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
I20260807 08:37:19.920477 13963 tablet_service.cc:1467] Processing CreateTablet for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a (DEFAULT_TABLE table=test-table [id=2d062fb1a44a44e2921da9f62d4b989e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:19.920607 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:19.915473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.10.207.1:34629)
0807 08:37:19.915745 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:19.915748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:19.915757 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:19.915794 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:19.915796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:19.915808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:19.916125 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:19.916193 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.916971 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.918637 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:19.919418 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:19.920354 (+ 936us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:19.920424 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:19.920449 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 08:37:19.920493 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":113,"threads_started":1}
I20260807 08:37:19.920872 13963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:19.920970 14175 tablet_service.cc:1467] Processing CreateTablet for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a (DEFAULT_TABLE table=test-table [id=2d062fb1a44a44e2921da9f62d4b989e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:19.921365 14175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:19.921336 13751 tablet_service.cc:1467] Processing CreateTablet for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a (DEFAULT_TABLE table=test-table [id=2d062fb1a44a44e2921da9f62d4b989e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:19.921670 13751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:19.923956 14314 tablet_bootstrap.cc:492] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Bootstrap starting.
I20260807 08:37:19.924606 14314 tablet_bootstrap.cc:654] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:19.924947 14314 log.cc:824] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:19.925731 14316 tablet_bootstrap.cc:492] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Bootstrap starting.
I20260807 08:37:19.925774 14314 tablet_bootstrap.cc:492] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: No bootstrap required, opened a new log
I20260807 08:37:19.925840 14314 ts_tablet_manager.cc:1397] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:19.926011 14318 tablet_bootstrap.cc:492] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Bootstrap starting.
I20260807 08:37:19.926788 14318 tablet_bootstrap.cc:654] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:19.926946 14316 tablet_bootstrap.cc:654] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:19.927122 14318 log.cc:824] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:19.927413 14316 log.cc:824] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:19.928028 14318 tablet_bootstrap.cc:492] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: No bootstrap required, opened a new log
I20260807 08:37:19.928143 14318 ts_tablet_manager.cc:1397] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:19.928097 14314 raft_consensus.cc:348] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:19.928275 14314 raft_consensus.cc:374] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:19.928313 14314 raft_consensus.cc:729] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Initialized, Role: FOLLOWER
I20260807 08:37:19.928440 14314 consensus_queue.cc:260] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:19.928723 14314 ts_tablet_manager.cc:1428] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:19.928912 14063 heartbeater.cc:503] Master 127.10.207.62:43719 was elected leader, sending a full tablet report...
I20260807 08:37:19.929889 14318 raft_consensus.cc:348] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:19.930017 14318 raft_consensus.cc:374] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:19.930042 14318 raft_consensus.cc:729] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2b305201c24ef2be1df3192dad1624, State: Initialized, Role: FOLLOWER
I20260807 08:37:19.930183 14318 consensus_queue.cc:260] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:19.930449 14318 ts_tablet_manager.cc:1428] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:37:19.930749 13856 heartbeater.cc:503] Master 127.10.207.62:43719 was elected leader, sending a full tablet report...
I20260807 08:37:19.931607 14316 tablet_bootstrap.cc:492] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: No bootstrap required, opened a new log
I20260807 08:37:19.931692 14316 ts_tablet_manager.cc:1397] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 08:37:19.933337 14316 raft_consensus.cc:348] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:19.933481 14316 raft_consensus.cc:374] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:19.933503 14316 raft_consensus.cc:729] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7153cd84417544629c89a9091820481a, State: Initialized, Role: FOLLOWER
I20260807 08:37:19.933641 14316 consensus_queue.cc:260] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:19.933954 14316 ts_tablet_manager.cc:1428] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:37:19.934046 14271 heartbeater.cc:503] Master 127.10.207.62:43719 was elected leader, sending a full tablet report...
W20260807 08:37:19.952385 14064 tablet.cc:2369] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:20.009747 13857 tablet.cc:2369] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:20.121855 14275 tablet.cc:2369] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:20.241032 14321 raft_consensus.cc:482] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:20.241202 14321 raft_consensus.cc:504] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:20.241652 14321 leader_election.cc:288] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7153cd84417544629c89a9091820481a (127.10.207.3:40539), fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629)
I20260807 08:37:20.245273 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.242018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:44825 (local address 127.10.207.3:40539)
0807 08:37:20.242092 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.242094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.242243 (+ 149us) server_negotiation.cc:407] Connection header received
0807 08:37:20.242323 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.242324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.242336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.242419 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:20.242688 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.243385 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.244464 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.244859 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.244928 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.244936 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.245146 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.245193 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:37:20.245299 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.241905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:40539 (local address 127.10.207.2:44825)
0807 08:37:20.242121 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.242136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.242229 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.242436 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.242438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.242441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:20.242578 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.242581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.243482 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.243483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.244353 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.244356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.244961 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.244962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.245052 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.245058 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.245119 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.245169 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
I20260807 08:37:20.245379 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.242085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:39333 (local address 127.10.207.1:34629)
0807 08:37:20.242144 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.242146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.242277 (+ 131us) server_negotiation.cc:407] Connection header received
0807 08:37:20.242369 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.242370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.242407 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.242472 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:20.242787 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.243461 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.244532 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.244930 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.245012 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.245023 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.245252 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.245295 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:37:20.245666 14326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.241990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:34629 (local address 127.10.207.2:39333)
0807 08:37:20.242150 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.242164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.242275 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.242507 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.242508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.242512 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:20.242693 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.242697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.243572 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.243573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.244387 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.244389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.245049 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.245050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.245144 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.245157 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.245231 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.245285 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260807 08:37:20.245990 14208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e4b27e6c5744ef8ed5ee2b9d67aa1a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7153cd84417544629c89a9091820481a" is_pre_election: true
I20260807 08:37:20.246228 14208 raft_consensus.cc:2454] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 0.
I20260807 08:37:20.246240 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e4b27e6c5744ef8ed5ee2b9d67aa1a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624" is_pre_election: true
I20260807 08:37:20.246448 13784 raft_consensus.cc:2454] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 0.
I20260807 08:37:20.246513 13902 leader_election.cc:302] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, 7153cd84417544629c89a9091820481a; no voters:
I20260807 08:37:20.246666 14321 raft_consensus.cc:2781] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:20.246732 14321 raft_consensus.cc:482] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:20.246762 14321 raft_consensus.cc:3037] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.247560 14321 raft_consensus.cc:504] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:20.247745 14321 leader_election.cc:288] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 election: Requested vote from peers 7153cd84417544629c89a9091820481a (127.10.207.3:40539), fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629)
I20260807 08:37:20.247938 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e4b27e6c5744ef8ed5ee2b9d67aa1a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624"
I20260807 08:37:20.248034 13784 raft_consensus.cc:3037] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.248785 13784 raft_consensus.cc:2454] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 1.
I20260807 08:37:20.248962 14208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e4b27e6c5744ef8ed5ee2b9d67aa1a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7153cd84417544629c89a9091820481a"
I20260807 08:37:20.249047 13903 leader_election.cc:302] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, fe2b305201c24ef2be1df3192dad1624; no voters:
I20260807 08:37:20.249086 14208 raft_consensus.cc:3037] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.249193 14321 raft_consensus.cc:2781] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:20.249405 14321 raft_consensus.cc:686] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 1 LEADER]: Becoming Leader. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Running, Role: LEADER
I20260807 08:37:20.249693 14321 consensus_queue.cc:237] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } }
I20260807 08:37:20.250070 14208 raft_consensus.cc:2454] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 1.
I20260807 08:37:20.250792 13550 catalog_manager.cc:5495] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2). New cstate: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } health_report { overall_health: HEALTHY } } }
Aug 07 08:37:20 dist-test-slave-h6n0 krb5kdc[13488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091839, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:20.304059 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.295527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43719 (local address 127.0.0.1:50470)
0807 08:37:20.295796 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.295809 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.295900 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.296097 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.296098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.296518 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:20.296603 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.296607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.297369 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.297371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.297995 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.297998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.298357 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.298358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.298417 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.299091 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:20.299096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:20.300732 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:20.302458 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:20.302467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:20.302470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:20.302803 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:20.303129 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:20.303130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:20.303130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:20.303198 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:20.303489 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:20.303498 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:20.303557 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.303636 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.303833 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":3,"thread_start_us":181,"threads_started":1}
I20260807 08:37:20.304057 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.295631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.10.207.62:43719)
0807 08:37:20.295726 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.295729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.295958 (+ 229us) server_negotiation.cc:407] Connection header received
0807 08:37:20.295993 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.295994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.296007 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.296059 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:20.296714 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.297233 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.298083 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.298269 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.298346 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.300828 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:20.300829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:20.300829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:20.300835 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:20.302328 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:20.302911 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:20.302915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:20.302915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:20.302981 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:20.303279 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:20.303280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:20.303281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:20.303417 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:20.303500 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.303675 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.303894 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":59}
I20260807 08:37:20.307013 13550 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:50470:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:37:20.307293 13550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:20.309226 13963 tablet_service.cc:1467] Processing CreateTablet for tablet 888b9257c00d4fa2a81cbd67dd680274 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a819d75b95840068cf137125b80a753]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:20.309301 14175 tablet_service.cc:1467] Processing CreateTablet for tablet 888b9257c00d4fa2a81cbd67dd680274 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a819d75b95840068cf137125b80a753]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:20.309280 13751 tablet_service.cc:1467] Processing CreateTablet for tablet 888b9257c00d4fa2a81cbd67dd680274 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a819d75b95840068cf137125b80a753]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:20.309415 13963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 888b9257c00d4fa2a81cbd67dd680274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:20.309523 14175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 888b9257c00d4fa2a81cbd67dd680274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:20.309523 13751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 888b9257c00d4fa2a81cbd67dd680274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:20.310813 14314 tablet_bootstrap.cc:492] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: Bootstrap starting.
I20260807 08:37:20.311265 14314 tablet_bootstrap.cc:654] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:20.311447 14318 tablet_bootstrap.cc:492] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: Bootstrap starting.
I20260807 08:37:20.311486 14316 tablet_bootstrap.cc:492] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: Bootstrap starting.
I20260807 08:37:20.311993 14318 tablet_bootstrap.cc:654] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:20.312178 14316 tablet_bootstrap.cc:654] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:20.312490 14314 tablet_bootstrap.cc:492] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: No bootstrap required, opened a new log
I20260807 08:37:20.312564 14314 ts_tablet_manager.cc:1397] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:20.312747 14318 tablet_bootstrap.cc:492] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: No bootstrap required, opened a new log
I20260807 08:37:20.312718 14314 raft_consensus.cc:348] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.312801 14318 ts_tablet_manager.cc:1397] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:20.312817 14314 raft_consensus.cc:374] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:20.312844 14314 raft_consensus.cc:729] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Initialized, Role: FOLLOWER
I20260807 08:37:20.312893 14314 consensus_queue.cc:260] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.313004 14316 tablet_bootstrap.cc:492] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: No bootstrap required, opened a new log
I20260807 08:37:20.313017 14314 ts_tablet_manager.cc:1428] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:20.312973 14318 raft_consensus.cc:348] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.313057 14318 raft_consensus.cc:374] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:20.313052 14316 ts_tablet_manager.cc:1397] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:20.313079 14318 raft_consensus.cc:729] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2b305201c24ef2be1df3192dad1624, State: Initialized, Role: FOLLOWER
I20260807 08:37:20.313124 14318 consensus_queue.cc:260] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.313109 14321 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.313225 14318 ts_tablet_manager.cc:1428] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 08:37:20.313231 14321 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:20.313221 14316 raft_consensus.cc:348] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.313308 14316 raft_consensus.cc:374] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:20.313333 14316 raft_consensus.cc:729] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7153cd84417544629c89a9091820481a, State: Initialized, Role: FOLLOWER
I20260807 08:37:20.313390 14316 consensus_queue.cc:260] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.313374 14322 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.313452 14322 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:20.313481 14316 ts_tablet_manager.cc:1428] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:20.313794 14323 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.313880 14323 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:20.356624 14321 raft_consensus.cc:482] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:20.356704 14321 raft_consensus.cc:504] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.356868 14321 leader_election.cc:288] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 7153cd84417544629c89a9091820481a (127.10.207.3:40539)
I20260807 08:37:20.357311 14208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888b9257c00d4fa2a81cbd67dd680274" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7153cd84417544629c89a9091820481a" is_pre_election: true
I20260807 08:37:20.357420 14208 raft_consensus.cc:2454] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 0.
I20260807 08:37:20.357366 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888b9257c00d4fa2a81cbd67dd680274" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624" is_pre_election: true
I20260807 08:37:20.357589 13784 raft_consensus.cc:2454] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 0.
I20260807 08:37:20.357590 13902 leader_election.cc:302] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, 7153cd84417544629c89a9091820481a; no voters:
I20260807 08:37:20.357749 14321 raft_consensus.cc:2781] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:20.357821 14321 raft_consensus.cc:482] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:20.357847 14321 raft_consensus.cc:3037] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.358809 14321 raft_consensus.cc:504] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.358987 14321 leader_election.cc:288] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 election: Requested vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 7153cd84417544629c89a9091820481a (127.10.207.3:40539)
I20260807 08:37:20.359220 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888b9257c00d4fa2a81cbd67dd680274" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624"
I20260807 08:37:20.359357 13784 raft_consensus.cc:3037] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.359378 14208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888b9257c00d4fa2a81cbd67dd680274" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7153cd84417544629c89a9091820481a"
I20260807 08:37:20.359481 14208 raft_consensus.cc:3037] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.360417 13784 raft_consensus.cc:2454] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 1.
I20260807 08:37:20.360417 14208 raft_consensus.cc:2454] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 1.
I20260807 08:37:20.360658 13902 leader_election.cc:302] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, 7153cd84417544629c89a9091820481a; no voters:
I20260807 08:37:20.360790 14321 raft_consensus.cc:2781] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:20.360853 14321 raft_consensus.cc:686] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [term 1 LEADER]: Becoming Leader. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Running, Role: LEADER
I20260807 08:37:20.360909 14321 consensus_queue.cc:237] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.361421 14329 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: TxnStatusTablet state changed. Reason: New leader 2ba3526d1b7c486184a08448c20ae18e. Latest consensus state: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.361549 14329 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:20.361819 14343 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:37:20.361986 14343 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:37:20.362067 13552 catalog_manager.cc:5495] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2). New cstate: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:20.376379 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.372789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.10.207.2:35659)
0807 08:37:20.372853 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.372855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.372897 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:37:20.373013 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.373014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.373032 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.373101 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:20.373322 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.373795 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.375178 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.375400 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.375487 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.375818 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.375987 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.376112 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.376190 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.376304 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:37:20.376365 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.372731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35659 (local address 127.0.0.1:35822)
0807 08:37:20.372839 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.372853 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.372921 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.373136 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.373137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.373139 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:20.373217 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.373222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.373893 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.373894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.375031 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.375034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.375519 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.375520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.375601 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.375621 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.376103 (+ 482us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.376113 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.376183 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.376265 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 08:37:20.378269 14208 raft_consensus.cc:1261] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Refusing update from remote peer 2ba3526d1b7c486184a08448c20ae18e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:20.378474 13784 raft_consensus.cc:1261] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Refusing update from remote peer 2ba3526d1b7c486184a08448c20ae18e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:20.378611 14321 consensus_queue.cc:1035] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:20.378854 14321 consensus_queue.cc:1035] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:20.379760 14323 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: TxnStatusTablet state changed. Reason: New leader 2ba3526d1b7c486184a08448c20ae18e. Latest consensus state: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.379845 14323 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:20.380151 14322 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: TxnStatusTablet state changed. Reason: New leader 2ba3526d1b7c486184a08448c20ae18e. Latest consensus state: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.380257 14322 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:20.380967 14321 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.381063 14321 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:20.381582 14323 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.381690 14323 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P 7153cd84417544629c89a9091820481a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:20.382115 14348 mvcc.cc:205] Tried to move back new op lower bound from 7315832178186649600 to 7315832178118799360. Current Snapshot: MvccSnapshot[applied={T|T < 7315832178186649600}]
I20260807 08:37:20.382252 14329 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.382334 14329 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P 2ba3526d1b7c486184a08448c20ae18e: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:20.382838 14322 tablet_replica.cc:439] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } } }
I20260807 08:37:20.382922 14322 tablet_replica.cc:442] T 888b9257c00d4fa2a81cbd67dd680274 P fe2b305201c24ef2be1df3192dad1624: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:20.383798 14352 mvcc.cc:205] Tried to move back new op lower bound from 7315832178186649600 to 7315832178118799360. Current Snapshot: MvccSnapshot[applied={T|T < 7315832178186649600}]
I20260807 08:37:20.386957 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.383802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.10.207.2:35659)
0807 08:37:20.383840 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.383841 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.383914 (+ 73us) server_negotiation.cc:407] Connection header received
0807 08:37:20.383984 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.383985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.384004 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.384074 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:20.384285 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.384839 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.386070 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.386281 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.386353 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.386600 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.386632 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.386696 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.386853 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.386877 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:37:20.392979 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.390282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:33811 (local address 127.10.207.2:35659)
0807 08:37:20.390326 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.390328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.390510 (+ 182us) server_negotiation.cc:407] Connection header received
0807 08:37:20.390630 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.390631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.390639 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.390693 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:20.390926 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.391371 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.392233 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.392415 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.392515 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.392732 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.392768 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.392831 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.392893 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.392910 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:37:20.393078 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.390196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35659 (local address 127.10.207.2:33811)
0807 08:37:20.390415 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.390428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.390505 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.390730 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.390731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.390733 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:20.390811 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.390813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.391498 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.391500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.392136 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.392138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.392531 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.392532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.392611 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.392624 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.392833 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.392835 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.392889 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.392932 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":117,"threads_started":1}
I20260807 08:37:20.395797 13784 raft_consensus.cc:1261] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Refusing update from remote peer 2ba3526d1b7c486184a08448c20ae18e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:20.395840 14208 raft_consensus.cc:1261] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Refusing update from remote peer 2ba3526d1b7c486184a08448c20ae18e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:20.396085 14364 consensus_queue.cc:1035] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:20.396255 14321 consensus_queue.cc:1035] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:20.397684 14347 mvcc.cc:205] Tried to move back new op lower bound from 7315832178259738624 to 7315832177663647744. Current Snapshot: MvccSnapshot[applied={T|T < 7315832178259738624}]
I20260807 08:37:20.412467 13550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:20.412598 13550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:20.413538 13550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Sending DeleteTablet for 3 replicas of tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a
I20260807 08:37:20.413914 13751 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:20 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:56350
I20260807 08:37:20.413919 14175 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:20 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:33248
I20260807 08:37:20.414191 13962 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:20 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:35810
I20260807 08:37:20.414404 14373 tablet_replica.cc:330] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: stopping tablet replica
I20260807 08:37:20.414438 14371 tablet_replica.cc:330] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: stopping tablet replica
I20260807 08:37:20.414590 14371 raft_consensus.cc:2229] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:20.414671 14371 raft_consensus.cc:2258] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:20.414769 14373 raft_consensus.cc:2229] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:20.414760 13550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:20.414923 13550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:20.415455 14371 ts_tablet_manager.cc:1905] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:20.415493 14373 raft_consensus.cc:2258] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:20.416435 14374 tablet_replica.cc:330] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: stopping tablet replica
I20260807 08:37:20.416632 13962 tablet_service.cc:1467] Processing CreateTablet for tablet 471b000b1bb3463780a580b4bd7b6e04 (DEFAULT_TABLE table=test-table [id=b5b9300546504efc95c3aa70942686df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:20.416805 14374 raft_consensus.cc:2229] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:20.416980 13962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471b000b1bb3463780a580b4bd7b6e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:20.416978 14374 raft_consensus.cc:2258] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:20.417049 14373 ts_tablet_manager.cc:1905] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:20.418038 14374 ts_tablet_manager.cc:1905] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:20.418993 14175 tablet_service.cc:1467] Processing CreateTablet for tablet 471b000b1bb3463780a580b4bd7b6e04 (DEFAULT_TABLE table=test-table [id=b5b9300546504efc95c3aa70942686df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:20.419147 14175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471b000b1bb3463780a580b4bd7b6e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:20.419490 14373 ts_tablet_manager.cc:1918] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:20.419615 14373 log.cc:1196] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/c4e4b27e6c5744ef8ed5ee2b9d67aa1a
I20260807 08:37:20.420147 14373 ts_tablet_manager.cc:1939] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 7153cd84417544629c89a9091820481a: Deleting consensus metadata
I20260807 08:37:20.420223 14374 ts_tablet_manager.cc:1918] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:20.420326 14374 log.cc:1196] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/c4e4b27e6c5744ef8ed5ee2b9d67aa1a
I20260807 08:37:20.420801 14374 ts_tablet_manager.cc:1939] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P 2ba3526d1b7c486184a08448c20ae18e: Deleting consensus metadata
I20260807 08:37:20.420905 14314 tablet_bootstrap.cc:492] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: Bootstrap starting.
I20260807 08:37:20.421391 14314 tablet_bootstrap.cc:654] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:20.421754 14316 tablet_bootstrap.cc:492] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: Bootstrap starting.
I20260807 08:37:20.422175 14314 tablet_bootstrap.cc:492] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: No bootstrap required, opened a new log
I20260807 08:37:20.422200 14316 tablet_bootstrap.cc:654] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:20.422230 14314 ts_tablet_manager.cc:1397] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:20.422405 14314 raft_consensus.cc:348] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.422485 14314 raft_consensus.cc:374] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:20.422513 14314 raft_consensus.cc:729] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Initialized, Role: FOLLOWER
I20260807 08:37:20.422595 14314 consensus_queue.cc:260] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.422705 14314 ts_tablet_manager.cc:1428] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:20.422813 13533 catalog_manager.cc:4841] TS 7153cd84417544629c89a9091820481a (127.10.207.3:40539): tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a (table test-table [id=2d062fb1a44a44e2921da9f62d4b989e]) successfully deleted
I20260807 08:37:20.422931 13533 catalog_manager.cc:4841] TS 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659): tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a (table test-table [id=2d062fb1a44a44e2921da9f62d4b989e]) successfully deleted
I20260807 08:37:20.423169 14316 tablet_bootstrap.cc:492] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: No bootstrap required, opened a new log
I20260807 08:37:20.423214 14316 ts_tablet_manager.cc:1397] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:20.423401 14316 raft_consensus.cc:348] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.423486 14316 raft_consensus.cc:374] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:20.423516 14316 raft_consensus.cc:729] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7153cd84417544629c89a9091820481a, State: Initialized, Role: FOLLOWER
I20260807 08:37:20.423578 14316 consensus_queue.cc:260] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.423681 14316 ts_tablet_manager.cc:1428] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:20.423889 13751 tablet_service.cc:1467] Processing CreateTablet for tablet 471b000b1bb3463780a580b4bd7b6e04 (DEFAULT_TABLE table=test-table [id=b5b9300546504efc95c3aa70942686df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:20.424014 13751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471b000b1bb3463780a580b4bd7b6e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:20.425319 14371 ts_tablet_manager.cc:1918] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:20.425393 14371 log.cc:1196] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/c4e4b27e6c5744ef8ed5ee2b9d67aa1a
I20260807 08:37:20.425401 14318 tablet_bootstrap.cc:492] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: Bootstrap starting.
I20260807 08:37:20.425699 14371 ts_tablet_manager.cc:1939] T c4e4b27e6c5744ef8ed5ee2b9d67aa1a P fe2b305201c24ef2be1df3192dad1624: Deleting consensus metadata
I20260807 08:37:20.425988 14318 tablet_bootstrap.cc:654] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:20.426512 13535 catalog_manager.cc:4841] TS fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629): tablet c4e4b27e6c5744ef8ed5ee2b9d67aa1a (table test-table [id=2d062fb1a44a44e2921da9f62d4b989e]) successfully deleted
I20260807 08:37:20.426792 14318 tablet_bootstrap.cc:492] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: No bootstrap required, opened a new log
I20260807 08:37:20.426851 14318 ts_tablet_manager.cc:1397] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:20.427003 14318 raft_consensus.cc:348] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.427089 14318 raft_consensus.cc:374] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:20.427120 14318 raft_consensus.cc:729] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2b305201c24ef2be1df3192dad1624, State: Initialized, Role: FOLLOWER
I20260807 08:37:20.427174 14318 consensus_queue.cc:260] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.427294 14318 ts_tablet_manager.cc:1428] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:20.714646 14323 raft_consensus.cc:482] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:20.714814 14323 raft_consensus.cc:504] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.715188 14323 leader_election.cc:288] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659)
I20260807 08:37:20.719170 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.715583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:36673 (local address 127.10.207.2:35659)
0807 08:37:20.715659 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.715662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.715715 (+ 53us) server_negotiation.cc:407] Connection header received
0807 08:37:20.715839 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.715840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.715852 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.715912 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:20.716140 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.716689 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.718232 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.718734 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.718806 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.718820 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.719039 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.719106 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:37:20.719167 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.715389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35659 (local address 127.10.207.3:36673)
0807 08:37:20.715595 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.715610 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.715732 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.715975 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.715976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.715978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:20.716046 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.716047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.716789 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.716790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.718126 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.718128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.718841 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.718843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.718908 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.718913 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.719025 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.719069 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":34,"thread_start_us":99,"threads_started":1}
I20260807 08:37:20.719415 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471b000b1bb3463780a580b4bd7b6e04" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba3526d1b7c486184a08448c20ae18e" is_pre_election: true
I20260807 08:37:20.719528 13998 raft_consensus.cc:2454] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 0.
I20260807 08:37:20.719739 14099 leader_election.cc:302] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, 7153cd84417544629c89a9091820481a; no voters:
I20260807 08:37:20.719789 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.715485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:41505 (local address 127.10.207.1:34629)
0807 08:37:20.715551 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.715554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.715570 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:37:20.715717 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.715719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.715731 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.715804 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:20.716021 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.716783 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.718854 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.719313 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.719402 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.719412 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.719640 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.719689 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:37:20.719866 14323 raft_consensus.cc:2781] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:20.719915 14323 raft_consensus.cc:482] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:20.719944 14323 raft_consensus.cc:3037] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.720083 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.715371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:34629 (local address 127.10.207.3:41505)
0807 08:37:20.715505 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.715520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.715615 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.715838 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.715839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.715842 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:20.715944 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.715947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.716852 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.716852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.718126 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.718128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.719459 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.719461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.719557 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.719562 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.719935 (+ 373us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.720003 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":84,"thread_start_us":69,"threads_started":1}
I20260807 08:37:20.720484 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471b000b1bb3463780a580b4bd7b6e04" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624" is_pre_election: true
I20260807 08:37:20.720593 13784 raft_consensus.cc:2454] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 0.
I20260807 08:37:20.720921 14323 raft_consensus.cc:504] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.721074 14323 leader_election.cc:288] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 election: Requested vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659)
I20260807 08:37:20.721242 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471b000b1bb3463780a580b4bd7b6e04" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624"
I20260807 08:37:20.721323 13784 raft_consensus.cc:3037] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.721318 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471b000b1bb3463780a580b4bd7b6e04" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba3526d1b7c486184a08448c20ae18e"
I20260807 08:37:20.721374 13998 raft_consensus.cc:3037] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:20.722103 13784 raft_consensus.cc:2454] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 1.
I20260807 08:37:20.722208 13998 raft_consensus.cc:2454] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 1.
I20260807 08:37:20.722314 14103 leader_election.cc:302] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7153cd84417544629c89a9091820481a, fe2b305201c24ef2be1df3192dad1624; no voters:
I20260807 08:37:20.722478 14323 raft_consensus.cc:2781] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:20.722669 14323 raft_consensus.cc:686] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 1 LEADER]: Becoming Leader. State: Replica: 7153cd84417544629c89a9091820481a, State: Running, Role: LEADER
I20260807 08:37:20.722764 14323 consensus_queue.cc:237] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:20.723649 13552 catalog_manager.cc:5495] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7153cd84417544629c89a9091820481a (127.10.207.3). New cstate: current_term: 1 leader_uuid: "7153cd84417544629c89a9091820481a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260807 08:37:20.780704 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.777801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.10.207.3:40539)
0807 08:37:20.777940 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.777942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.777952 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:20.777999 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.778000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.778021 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.778103 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:20.778456 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.778991 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.779912 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.780109 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.780166 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.780345 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.780454 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.780517 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.780576 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.780594 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260807 08:37:20.786166 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.783716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:56655 (local address 127.10.207.2:35659)
0807 08:37:20.783801 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.783803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.783936 (+ 133us) server_negotiation.cc:407] Connection header received
0807 08:37:20.784000 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.784001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.784012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.784088 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:20.784234 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.784689 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.785444 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.785634 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.785715 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.785911 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.785950 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.786015 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.786071 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.786089 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 08:37:20.786213 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.783589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35659 (local address 127.10.207.3:56655)
0807 08:37:20.783844 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.783856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.783946 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.784093 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.784094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.784096 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:20.784156 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.784157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.784778 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.784779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.785346 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.785348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.785740 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.785741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.785807 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.785819 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.786019 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.786021 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.786068 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.786109 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":142,"threads_started":1}
I20260807 08:37:20.788659 13784 raft_consensus.cc:1261] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Refusing update from remote peer 7153cd84417544629c89a9091820481a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:20.788734 13998 raft_consensus.cc:1261] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Refusing update from remote peer 7153cd84417544629c89a9091820481a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:20.789012 14381 consensus_queue.cc:1035] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:20.789280 14323 consensus_queue.cc:1035] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:20.798657 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.795954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:50829 (local address 127.10.207.3:40539)
0807 08:37:20.796121 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:20.796124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:20.796133 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:20.796157 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:20.796157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:20.796167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:20.796233 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:20.796425 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.796983 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.797810 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.798055 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.798144 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.798334 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.798366 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.798427 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:20.798516 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 08:37:20.798580 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260807 08:37:20.798686 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:20.795864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:40539 (local address 127.10.207.2:50829)
0807 08:37:20.795912 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:20.795927 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:20.795972 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:20.796252 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:20.796253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:20.796255 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:20.796312 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.796316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.797075 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.797076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.797696 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:20.797698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:20.798146 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:20.798147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:20.798210 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:20.798221 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:20.798445 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:20.798447 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:20.798494 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:37:20.798584 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 08:37:20.798990 14348 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:37:22.321080 13550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:22.321249 13550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:22.322399 13550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Sending DeleteTablet for 3 replicas of tablet 471b000b1bb3463780a580b4bd7b6e04
I20260807 08:37:22.322741 13962 tablet_service.cc:1514] Processing DeleteTablet for tablet 471b000b1bb3463780a580b4bd7b6e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:35810
I20260807 08:37:22.322751 13751 tablet_service.cc:1514] Processing DeleteTablet for tablet 471b000b1bb3463780a580b4bd7b6e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:56350
I20260807 08:37:22.322994 14428 tablet_replica.cc:330] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: stopping tablet replica
I20260807 08:37:22.322997 14175 tablet_service.cc:1514] Processing DeleteTablet for tablet 471b000b1bb3463780a580b4bd7b6e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:33248
I20260807 08:37:22.323104 14428 raft_consensus.cc:2229] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:22.323213 14428 raft_consensus.cc:2258] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:22.323225 14430 tablet_replica.cc:330] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: stopping tablet replica
I20260807 08:37:22.323247 14429 tablet_replica.cc:330] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: stopping tablet replica
I20260807 08:37:22.323307 14429 raft_consensus.cc:2229] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:22.323360 14430 raft_consensus.cc:2229] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:22.323372 14429 raft_consensus.cc:2258] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:22.323521 14430 raft_consensus.cc:2258] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:22.323684 14429 ts_tablet_manager.cc:1905] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:22.323706 14428 ts_tablet_manager.cc:1905] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:22.323855 14430 ts_tablet_manager.cc:1905] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:22.325650 14429 ts_tablet_manager.cc:1918] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:37:22.325650 14428 ts_tablet_manager.cc:1918] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:37:22.325726 14428 log.cc:1196] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/471b000b1bb3463780a580b4bd7b6e04
I20260807 08:37:22.325845 14429 log.cc:1196] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/471b000b1bb3463780a580b4bd7b6e04
I20260807 08:37:22.325994 14430 ts_tablet_manager.cc:1918] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:37:22.326004 14428 ts_tablet_manager.cc:1939] T 471b000b1bb3463780a580b4bd7b6e04 P 2ba3526d1b7c486184a08448c20ae18e: Deleting consensus metadata
I20260807 08:37:22.326138 14430 log.cc:1196] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/471b000b1bb3463780a580b4bd7b6e04
I20260807 08:37:22.326323 14429 ts_tablet_manager.cc:1939] T 471b000b1bb3463780a580b4bd7b6e04 P fe2b305201c24ef2be1df3192dad1624: Deleting consensus metadata
I20260807 08:37:22.326421 13533 catalog_manager.cc:4841] TS 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659): tablet 471b000b1bb3463780a580b4bd7b6e04 (table test-table [id=b5b9300546504efc95c3aa70942686df]) successfully deleted
I20260807 08:37:22.326510 14430 ts_tablet_manager.cc:1939] T 471b000b1bb3463780a580b4bd7b6e04 P 7153cd84417544629c89a9091820481a: Deleting consensus metadata
I20260807 08:37:22.326900 13535 catalog_manager.cc:4841] TS fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629): tablet 471b000b1bb3463780a580b4bd7b6e04 (table test-table [id=b5b9300546504efc95c3aa70942686df]) successfully deleted
I20260807 08:37:22.326984 13533 catalog_manager.cc:4841] TS 7153cd84417544629c89a9091820481a (127.10.207.3:40539): tablet 471b000b1bb3463780a580b4bd7b6e04 (table test-table [id=b5b9300546504efc95c3aa70942686df]) successfully deleted
Aug 07 08:37:22 dist-test-slave-h6n0 krb5kdc[13488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091842, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:37:22 dist-test-slave-h6n0 krb5kdc[13488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091842, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:22.356992 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:22.345455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50476 (local address 127.10.207.62:43719)
0807 08:37:22.345688 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:22.345690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:22.345700 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:22.345791 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:22.345792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:22.345811 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:22.345891 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:22.346482 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.347191 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.347999 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.348218 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.348310 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:22.354293 (+ 5983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:22.354295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:22.354295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:22.354301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:22.355830 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:22.356238 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:22.356238 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:22.356239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:22.356305 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:22.356511 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:22.356511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:22.356512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:22.356616 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:22.356696 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:22.356843 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:37:22.356921 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":136,"threads_started":1}
I20260807 08:37:22.358779 13550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:22.358978 13550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:22.360796 13751 tablet_service.cc:1467] Processing CreateTablet for tablet bf2ec4a684694799a2035c251300330a (DEFAULT_TABLE table=test-table [id=9a54e575196e4f079fb20b0d7a0ba084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:22.360855 13962 tablet_service.cc:1467] Processing CreateTablet for tablet bf2ec4a684694799a2035c251300330a (DEFAULT_TABLE table=test-table [id=9a54e575196e4f079fb20b0d7a0ba084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:22.360900 14175 tablet_service.cc:1467] Processing CreateTablet for tablet bf2ec4a684694799a2035c251300330a (DEFAULT_TABLE table=test-table [id=9a54e575196e4f079fb20b0d7a0ba084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:22.360953 13751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2ec4a684694799a2035c251300330a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:22.360978 13962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2ec4a684694799a2035c251300330a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:22.361053 14175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2ec4a684694799a2035c251300330a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:22.362437 14444 tablet_bootstrap.cc:492] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: Bootstrap starting.
I20260807 08:37:22.362830 14447 tablet_bootstrap.cc:492] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: Bootstrap starting.
I20260807 08:37:22.362840 14445 tablet_bootstrap.cc:492] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: Bootstrap starting.
I20260807 08:37:22.362963 14444 tablet_bootstrap.cc:654] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:22.363267 14447 tablet_bootstrap.cc:654] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:22.363298 14445 tablet_bootstrap.cc:654] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:22.363845 14444 tablet_bootstrap.cc:492] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: No bootstrap required, opened a new log
I20260807 08:37:22.363942 14444 ts_tablet_manager.cc:1397] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:22.364023 14447 tablet_bootstrap.cc:492] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: No bootstrap required, opened a new log
I20260807 08:37:22.364104 14447 ts_tablet_manager.cc:1397] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:22.364126 14444 raft_consensus.cc:348] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.364214 14444 raft_consensus.cc:374] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:22.364243 14444 raft_consensus.cc:729] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7153cd84417544629c89a9091820481a, State: Initialized, Role: FOLLOWER
I20260807 08:37:22.364297 14444 consensus_queue.cc:260] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.364312 14447 raft_consensus.cc:348] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.364395 14444 ts_tablet_manager.cc:1428] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:22.364416 14447 raft_consensus.cc:374] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:22.364442 14447 raft_consensus.cc:729] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2b305201c24ef2be1df3192dad1624, State: Initialized, Role: FOLLOWER
I20260807 08:37:22.364509 14447 consensus_queue.cc:260] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.364722 14447 ts_tablet_manager.cc:1428] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:22.365031 14445 tablet_bootstrap.cc:492] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: No bootstrap required, opened a new log
I20260807 08:37:22.365130 14445 ts_tablet_manager.cc:1397] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:22.365309 14445 raft_consensus.cc:348] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.365397 14445 raft_consensus.cc:374] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:22.365424 14445 raft_consensus.cc:729] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Initialized, Role: FOLLOWER
I20260807 08:37:22.365505 14445 consensus_queue.cc:260] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.365648 14445 ts_tablet_manager.cc:1428] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:22.471127 14412 raft_consensus.cc:482] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:22.471324 14412 raft_consensus.cc:504] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.471630 14412 leader_election.cc:288] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 7153cd84417544629c89a9091820481a (127.10.207.3:40539)
I20260807 08:37:22.471938 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2ec4a684694799a2035c251300330a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624" is_pre_election: true
I20260807 08:37:22.471980 14208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2ec4a684694799a2035c251300330a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7153cd84417544629c89a9091820481a" is_pre_election: true
I20260807 08:37:22.472072 13784 raft_consensus.cc:2454] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 0.
I20260807 08:37:22.472110 14208 raft_consensus.cc:2454] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 0.
I20260807 08:37:22.472318 13903 leader_election.cc:302] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, fe2b305201c24ef2be1df3192dad1624; no voters:
I20260807 08:37:22.472443 14412 raft_consensus.cc:2781] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:22.472499 14412 raft_consensus.cc:482] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:22.472529 14412 raft_consensus.cc:3037] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:22.473253 14412 raft_consensus.cc:504] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.473416 14412 leader_election.cc:288] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 election: Requested vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 7153cd84417544629c89a9091820481a (127.10.207.3:40539)
I20260807 08:37:22.473625 14208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2ec4a684694799a2035c251300330a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7153cd84417544629c89a9091820481a"
I20260807 08:37:22.473645 13784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2ec4a684694799a2035c251300330a" candidate_uuid: "2ba3526d1b7c486184a08448c20ae18e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624"
I20260807 08:37:22.473731 14208 raft_consensus.cc:3037] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:22.473733 13784 raft_consensus.cc:3037] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:22.474382 14208 raft_consensus.cc:2454] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 1.
I20260807 08:37:22.474669 13902 leader_election.cc:302] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, 7153cd84417544629c89a9091820481a; no voters:
I20260807 08:37:22.474745 13784 raft_consensus.cc:2454] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba3526d1b7c486184a08448c20ae18e in term 1.
I20260807 08:37:22.474794 14412 raft_consensus.cc:2781] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:22.474852 14412 raft_consensus.cc:686] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 1 LEADER]: Becoming Leader. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Running, Role: LEADER
I20260807 08:37:22.474915 14412 consensus_queue.cc:237] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.475672 13552 catalog_manager.cc:5495] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2). New cstate: current_term: 1 leader_uuid: "2ba3526d1b7c486184a08448c20ae18e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:22.487300 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:22.484606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.10.207.2:35659)
0807 08:37:22.484762 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:22.484765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:22.484772 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:37:22.484795 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:22.484796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:22.484806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:22.484879 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:22.485098 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.485762 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.486618 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.486854 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.486921 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:22.487058 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:22.487097 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:22.487143 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:22.487204 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 08:37:22.487224 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260807 08:37:22.490293 13784 raft_consensus.cc:1261] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Refusing update from remote peer 2ba3526d1b7c486184a08448c20ae18e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:22.490300 14208 raft_consensus.cc:1261] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Refusing update from remote peer 2ba3526d1b7c486184a08448c20ae18e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:22.490547 14456 consensus_queue.cc:1035] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:22.490799 14456 consensus_queue.cc:1035] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:22.502418 13552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:22.502626 13552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:22.503459 13552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Sending DeleteTablet for 3 replicas of tablet bf2ec4a684694799a2035c251300330a
I20260807 08:37:22.503835 13751 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2ec4a684694799a2035c251300330a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:56350
I20260807 08:37:22.503912 13962 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2ec4a684694799a2035c251300330a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:35810
I20260807 08:37:22.503947 14175 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2ec4a684694799a2035c251300330a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:33248
I20260807 08:37:22.503984 14429 tablet_replica.cc:330] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: stopping tablet replica
I20260807 08:37:22.504027 14428 tablet_replica.cc:330] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: stopping tablet replica
I20260807 08:37:22.504061 14429 raft_consensus.cc:2229] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:22.504117 14429 raft_consensus.cc:2258] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:22.504220 14428 raft_consensus.cc:2229] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:22.504333 14428 raft_consensus.cc:2258] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:22.504449 14430 tablet_replica.cc:330] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: stopping tablet replica
I20260807 08:37:22.504393 13552 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:37:22.504528 14430 raft_consensus.cc:2229] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:22.504567 14430 raft_consensus.cc:2258] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 08:37:22.504576 13552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:22.504984 14428 ts_tablet_manager.cc:1905] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:22.505105 14430 ts_tablet_manager.cc:1905] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:22.506152 13962 tablet_service.cc:1467] Processing CreateTablet for tablet ae1987d65afc476fa699b0188db1ce51 (DEFAULT_TABLE table=test-table [id=38deeb61b69144dca8bcd0f7c93312fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:22.506201 13751 tablet_service.cc:1467] Processing CreateTablet for tablet ae1987d65afc476fa699b0188db1ce51 (DEFAULT_TABLE table=test-table [id=38deeb61b69144dca8bcd0f7c93312fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:22.506285 13962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1987d65afc476fa699b0188db1ce51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:22.506319 13751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1987d65afc476fa699b0188db1ce51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:22.506966 14430 ts_tablet_manager.cc:1918] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:22.507021 14430 log.cc:1196] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/bf2ec4a684694799a2035c251300330a
I20260807 08:37:22.507239 14428 ts_tablet_manager.cc:1918] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:22.507299 14430 ts_tablet_manager.cc:1939] T bf2ec4a684694799a2035c251300330a P 7153cd84417544629c89a9091820481a: Deleting consensus metadata
I20260807 08:37:22.507324 14428 log.cc:1196] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/bf2ec4a684694799a2035c251300330a
I20260807 08:37:22.507601 14428 ts_tablet_manager.cc:1939] T bf2ec4a684694799a2035c251300330a P 2ba3526d1b7c486184a08448c20ae18e: Deleting consensus metadata
I20260807 08:37:22.507669 13533 catalog_manager.cc:4841] TS 7153cd84417544629c89a9091820481a (127.10.207.3:40539): tablet bf2ec4a684694799a2035c251300330a (table test-table [id=9a54e575196e4f079fb20b0d7a0ba084]) successfully deleted
I20260807 08:37:22.507905 14175 tablet_service.cc:1467] Processing CreateTablet for tablet ae1987d65afc476fa699b0188db1ce51 (DEFAULT_TABLE table=test-table [id=38deeb61b69144dca8bcd0f7c93312fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:22.507983 14429 ts_tablet_manager.cc:1905] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:22.508060 14175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1987d65afc476fa699b0188db1ce51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:22.508106 14445 tablet_bootstrap.cc:492] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: Bootstrap starting.
I20260807 08:37:22.508422 14445 tablet_bootstrap.cc:654] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:22.509069 14445 tablet_bootstrap.cc:492] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: No bootstrap required, opened a new log
I20260807 08:37:22.509131 14445 ts_tablet_manager.cc:1397] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:22.509231 14445 raft_consensus.cc:348] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.509333 14445 raft_consensus.cc:374] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:22.509358 14445 raft_consensus.cc:729] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba3526d1b7c486184a08448c20ae18e, State: Initialized, Role: FOLLOWER
I20260807 08:37:22.509439 14445 consensus_queue.cc:260] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.509528 14444 tablet_bootstrap.cc:492] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: Bootstrap starting.
I20260807 08:37:22.509568 14445 ts_tablet_manager.cc:1428] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:22.509668 14429 ts_tablet_manager.cc:1918] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:22.509732 14429 log.cc:1196] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/bf2ec4a684694799a2035c251300330a
I20260807 08:37:22.510000 14429 ts_tablet_manager.cc:1939] T bf2ec4a684694799a2035c251300330a P fe2b305201c24ef2be1df3192dad1624: Deleting consensus metadata
I20260807 08:37:22.510030 14447 tablet_bootstrap.cc:492] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624: Bootstrap starting.
I20260807 08:37:22.510054 14444 tablet_bootstrap.cc:654] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:22.510360 13535 catalog_manager.cc:4841] TS fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629): tablet bf2ec4a684694799a2035c251300330a (table test-table [id=9a54e575196e4f079fb20b0d7a0ba084]) successfully deleted
I20260807 08:37:22.510990 14447 tablet_bootstrap.cc:654] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:22.511693 14444 tablet_bootstrap.cc:492] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: No bootstrap required, opened a new log
I20260807 08:37:22.511755 14444 ts_tablet_manager.cc:1397] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:22.511875 14444 raft_consensus.cc:348] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.511957 14444 raft_consensus.cc:374] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:22.511986 14444 raft_consensus.cc:729] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7153cd84417544629c89a9091820481a, State: Initialized, Role: FOLLOWER
I20260807 08:37:22.512041 14444 consensus_queue.cc:260] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.512156 14444 ts_tablet_manager.cc:1428] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:22.512416 14447 tablet_bootstrap.cc:492] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624: No bootstrap required, opened a new log
I20260807 08:37:22.512470 14447 ts_tablet_manager.cc:1397] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:22.512641 14447 raft_consensus.cc:348] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.512717 14447 raft_consensus.cc:374] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:22.512735 14447 raft_consensus.cc:729] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2b305201c24ef2be1df3192dad1624, State: Initialized, Role: FOLLOWER
I20260807 08:37:22.512782 14447 consensus_queue.cc:260] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.512871 14447 ts_tablet_manager.cc:1428] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:22.513479 13533 catalog_manager.cc:4841] TS 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659): tablet bf2ec4a684694799a2035c251300330a (table test-table [id=9a54e575196e4f079fb20b0d7a0ba084]) successfully deleted
I20260807 08:37:22.539552 14401 raft_consensus.cc:482] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:22.539686 14401 raft_consensus.cc:504] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.539896 14401 leader_election.cc:288] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659)
I20260807 08:37:22.540186 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1987d65afc476fa699b0188db1ce51" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba3526d1b7c486184a08448c20ae18e" is_pre_election: true
I20260807 08:37:22.540230 13783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1987d65afc476fa699b0188db1ce51" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624" is_pre_election: true
I20260807 08:37:22.540326 13998 raft_consensus.cc:2454] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 0.
I20260807 08:37:22.540388 13783 raft_consensus.cc:2454] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 0.
I20260807 08:37:22.540529 14099 leader_election.cc:302] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, 7153cd84417544629c89a9091820481a; no voters:
I20260807 08:37:22.540654 14401 raft_consensus.cc:2781] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:22.540714 14401 raft_consensus.cc:482] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:22.540745 14401 raft_consensus.cc:3037] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:22.541750 14401 raft_consensus.cc:504] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.541913 14401 leader_election.cc:288] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 election: Requested vote from peers fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629), 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659)
I20260807 08:37:22.542064 13783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1987d65afc476fa699b0188db1ce51" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2b305201c24ef2be1df3192dad1624"
I20260807 08:37:22.542081 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1987d65afc476fa699b0188db1ce51" candidate_uuid: "7153cd84417544629c89a9091820481a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba3526d1b7c486184a08448c20ae18e"
I20260807 08:37:22.542168 13998 raft_consensus.cc:3037] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:22.542171 13783 raft_consensus.cc:3037] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:22.543116 13998 raft_consensus.cc:2454] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 1.
I20260807 08:37:22.543280 13783 raft_consensus.cc:2454] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7153cd84417544629c89a9091820481a in term 1.
I20260807 08:37:22.543291 14099 leader_election.cc:302] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba3526d1b7c486184a08448c20ae18e, 7153cd84417544629c89a9091820481a; no voters:
I20260807 08:37:22.543394 14401 raft_consensus.cc:2781] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:22.543458 14401 raft_consensus.cc:686] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 1 LEADER]: Becoming Leader. State: Replica: 7153cd84417544629c89a9091820481a, State: Running, Role: LEADER
I20260807 08:37:22.543519 14401 consensus_queue.cc:237] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } }
I20260807 08:37:22.544255 13552 catalog_manager.cc:5495] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7153cd84417544629c89a9091820481a (127.10.207.3). New cstate: current_term: 1 leader_uuid: "7153cd84417544629c89a9091820481a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7153cd84417544629c89a9091820481a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260807 08:37:22.553715 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:22.551230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33268 (local address 127.10.207.3:40539)
0807 08:37:22.551426 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:22.551428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:22.551434 (+ 6us) server_negotiation.cc:407] Connection header received
0807 08:37:22.551452 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:22.551453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:22.551463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:22.551525 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:22.551705 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.552183 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.553047 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:22.553235 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:22.553294 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:22.553453 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:22.553481 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:22.553532 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:22.553597 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:37:22.553614 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
I20260807 08:37:22.556473 13998 raft_consensus.cc:1261] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Refusing update from remote peer 7153cd84417544629c89a9091820481a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:22.556524 13783 raft_consensus.cc:1261] T ae1987d65afc476fa699b0188db1ce51 P fe2b305201c24ef2be1df3192dad1624 [term 1 FOLLOWER]: Refusing update from remote peer 7153cd84417544629c89a9091820481a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:22.556753 14422 consensus_queue.cc:1035] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba3526d1b7c486184a08448c20ae18e" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:22.556912 14401 consensus_queue.cc:1035] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2b305201c24ef2be1df3192dad1624" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:37:22.561968 14407 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:37:24.087208 13549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:24.087364 13549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:24.088532 13549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07a64e43f5b243d3a5c876c2b0ec9177: Sending DeleteTablet for 3 replicas of tablet ae1987d65afc476fa699b0188db1ce51
I20260807 08:37:24.088878 14175 tablet_service.cc:1514] Processing DeleteTablet for tablet ae1987d65afc476fa699b0188db1ce51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:24 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:33248
I20260807 08:37:24.088994 13962 tablet_service.cc:1514] Processing DeleteTablet for tablet ae1987d65afc476fa699b0188db1ce51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:24 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:35810
I20260807 08:37:24.089042 13751 tablet_service.cc:1514] Processing DeleteTablet for tablet ae1987d65afc476fa699b0188db1ce51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:24 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:56350
I20260807 08:37:24.089120 14506 tablet_replica.cc:330] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: stopping tablet replica
I20260807 08:37:24.089181 14506 raft_consensus.cc:2229] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:24.089260 14506 raft_consensus.cc:2258] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:24.089519 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 13660
I20260807 08:37:24.089680 14506 ts_tablet_manager.cc:1905] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:24.098096 14506 ts_tablet_manager.cc:1918] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:37:24.098191 14506 log.cc:1196] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/ae1987d65afc476fa699b0188db1ce51
I20260807 08:37:24.098490 14506 ts_tablet_manager.cc:1939] T ae1987d65afc476fa699b0188db1ce51 P 7153cd84417544629c89a9091820481a: Deleting consensus metadata
I20260807 08:37:24.098899 14508 tablet_replica.cc:330] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: stopping tablet replica
I20260807 08:37:24.098987 14508 raft_consensus.cc:2229] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:24.098949 13533 catalog_manager.cc:4841] TS 7153cd84417544629c89a9091820481a (127.10.207.3:40539): tablet ae1987d65afc476fa699b0188db1ce51 (table test-table [id=38deeb61b69144dca8bcd0f7c93312fd]) successfully deleted
I20260807 08:37:24.099048 14508 raft_consensus.cc:2258] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:24.099848 14508 ts_tablet_manager.cc:1905] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:24.101354 14508 ts_tablet_manager.cc:1918] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:37:24.101436 14508 log.cc:1196] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/ae1987d65afc476fa699b0188db1ce51
I20260807 08:37:24.101697 14508 ts_tablet_manager.cc:1939] T ae1987d65afc476fa699b0188db1ce51 P 2ba3526d1b7c486184a08448c20ae18e: Deleting consensus metadata
I20260807 08:37:24.102093 13533 catalog_manager.cc:4841] TS 2ba3526d1b7c486184a08448c20ae18e (127.10.207.2:35659): tablet ae1987d65afc476fa699b0188db1ce51 (table test-table [id=38deeb61b69144dca8bcd0f7c93312fd]) successfully deleted
W20260807 08:37:24.105752 13535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:34629 (error 108)
I20260807 08:37:24.106074 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 13875
I20260807 08:37:24.116982 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:24.106257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:34629 (local address 127.0.0.1:56360)
0807 08:37:24.116790 (+ 10533us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:24.116904 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:34629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10480,"thread_start_us":69,"threads_started":1}
W20260807 08:37:24.117168 13535 catalog_manager.cc:4584] TS fe2b305201c24ef2be1df3192dad1624 (127.10.207.1:34629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae1987d65afc476fa699b0188db1ce51: Network error: Client connection negotiation failed: client connection to 127.10.207.1:34629: connect: Connection refused (error 111)
I20260807 08:37:24.124418 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 14077
I20260807 08:37:24.139329 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 13511
2026-08-07T08:37:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6417 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:24 dist-test-slave-h6n0 krb5kdc[14517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:24 dist-test-slave-h6n0 krb5kdc[14517](info): set up 2 sockets
Aug 07 08:37:24 dist-test-slave-h6n0 krb5kdc[14517](info): commencing operation
krb5kdc: starting...
W20260807 08:37:25.435789 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:25 dist-test-slave-h6n0 krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:25.570013 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:37215
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:37215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:25.656143 14537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:25.656375 14537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:25.656401 14537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:25.657507 14537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:25.657557 14537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:25.657574 14537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:25.657589 14537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:25.657603 14537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:25.659162 14537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:37215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:37215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:25.659672 14537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:25.659966 14537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:25.666090 14543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:25.666090 14546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:25.666144 14537 server_base.cc:1034] running on GCE node
W20260807 08:37:25.666260 14542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:25.666741 14537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:25.667073 14537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:25.668242 14537 hybrid_clock.cc:648] HybridClock initialized: now 1786091845668223 us; error 43 us; skew 500 ppm
Aug 07 08:37:25 dist-test-slave-h6n0 krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091845, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:25.671404 14537 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:25.671965 14537 webserver.cc:460] Webserver started at http://127.10.207.62:36871/ using document root <none> and password file <none>
I20260807 08:37:25.672243 14537 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:25.672320 14537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:25.672485 14537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:25.673735 14537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "d18d74124bca4383a8494d1272ec4a07"
format_stamp: "Formatted at 2026-08-07 08:37:25 on dist-test-slave-h6n0"
server_key: "5e01448a5aa24181c81dad834bc44033"
server_key_iv: "bf1ce34595db17eed1acff36c4cc4a29"
server_key_version: "encryptionkey@0"
I20260807 08:37:25.674191 14537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "d18d74124bca4383a8494d1272ec4a07"
format_stamp: "Formatted at 2026-08-07 08:37:25 on dist-test-slave-h6n0"
server_key: "5e01448a5aa24181c81dad834bc44033"
server_key_iv: "bf1ce34595db17eed1acff36c4cc4a29"
server_key_version: "encryptionkey@0"
I20260807 08:37:25.675871 14537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 08:37:25.676820 14555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:25.677063 14537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:25.677210 14537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "d18d74124bca4383a8494d1272ec4a07"
format_stamp: "Formatted at 2026-08-07 08:37:25 on dist-test-slave-h6n0"
server_key: "5e01448a5aa24181c81dad834bc44033"
server_key_iv: "bf1ce34595db17eed1acff36c4cc4a29"
server_key_version: "encryptionkey@0"
I20260807 08:37:25.677295 14537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:25.702020 14537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:25.704671 14537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:25.704862 14537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:25.710330 14537 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:37215
I20260807 08:37:25.710410 14633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:37215 every 8 connection(s)
I20260807 08:37:25.710896 14537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:25.711964 14634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:25.714455 14634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07: Bootstrap starting.
I20260807 08:37:25.715113 14634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:25.715440 14634 log.cc:824] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:25.715881 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 14537
I20260807 08:37:25.715958 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:25.716149 11068 external_mini_cluster.cc:1428] Setting key 742b6ea070886babe23787a961ee6a19
I20260807 08:37:25.716437 14634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07: No bootstrap required, opened a new log
I20260807 08:37:25.719105 14634 raft_consensus.cc:348] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d74124bca4383a8494d1272ec4a07" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37215 } }
I20260807 08:37:25.719306 14634 raft_consensus.cc:374] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:25.719362 14634 raft_consensus.cc:729] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18d74124bca4383a8494d1272ec4a07, State: Initialized, Role: FOLLOWER
I20260807 08:37:25.719521 14634 consensus_queue.cc:260] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d74124bca4383a8494d1272ec4a07" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37215 } }
I20260807 08:37:25.719617 14634 raft_consensus.cc:388] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:25.719666 14634 raft_consensus.cc:482] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:25.719717 14634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:25.720438 14634 raft_consensus.cc:504] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d74124bca4383a8494d1272ec4a07" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37215 } }
I20260807 08:37:25.720569 14634 leader_election.cc:302] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d18d74124bca4383a8494d1272ec4a07; no voters:
I20260807 08:37:25.720800 14634 leader_election.cc:288] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:25.720860 14641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:25.721029 14641 raft_consensus.cc:686] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [term 1 LEADER]: Becoming Leader. State: Replica: d18d74124bca4383a8494d1272ec4a07, State: Running, Role: LEADER
I20260807 08:37:25.721136 14634 sys_catalog.cc:564] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:25.721246 14641 consensus_queue.cc:237] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d74124bca4383a8494d1272ec4a07" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37215 } }
I20260807 08:37:25.721760 14643 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d18d74124bca4383a8494d1272ec4a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d74124bca4383a8494d1272ec4a07" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37215 } } }
I20260807 08:37:25.721865 14643 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:25.722417 14647 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d18d74124bca4383a8494d1272ec4a07. Latest consensus state: current_term: 1 leader_uuid: "d18d74124bca4383a8494d1272ec4a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d74124bca4383a8494d1272ec4a07" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37215 } } }
I20260807 08:37:25.722635 14647 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07 [sys.catalog]: This master's current role is: LEADER
W20260807 08:37:25.723374 14665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:25.723425 14665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:25.723500 14658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:25.723695 14658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:37:25 dist-test-slave-h6n0 krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:25.725679 14658 catalog_manager.cc:1347] Generated new cluster ID: c1cd70bc6edf4e95b80e3d6d8604199c
I20260807 08:37:25.725733 14658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:25.730131 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:25.716824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.10.207.62:37215)
0807 08:37:25.717078 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:25.717087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:25.717113 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:37:25.718111 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:25.718119 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:25.718167 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:25.718310 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:25.719044 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:25.720493 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:25.721451 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:25.722052 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:25.722188 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:25.725346 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:25.725350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:25.725350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:25.725363 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:25.727547 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:25.728009 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:25.728012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:25.728012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:25.728139 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:25.728319 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:25.728320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:25.728320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:25.728621 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:25.729265 (+ 644us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:25.729752 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 08:37:25.729902 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":103,"threads_started":1}
I20260807 08:37:25.750321 14658 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:25.750963 14658 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:25.754999 14658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d18d74124bca4383a8494d1272ec4a07: Generated new TSK 0
I20260807 08:37:25.755245 14658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
W20260807 08:37:26.588474 11068 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.207.1: real 0.826s user 0.005s sys 0.000s
I20260807 08:37:26.588824 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37215
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:26.674335 14672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:26.674592 14672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:26.674621 14672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:26.675721 14672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:26.675761 14672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:26.675787 14672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:26.677124 14672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:26.677299 14672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:26.677475 14672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:26.682811 14685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:26.682889 14679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:26.682889 14680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:26.683228 14672 server_base.cc:1034] running on GCE node
I20260807 08:37:26.683503 14672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:26.683765 14672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:26.684927 14672 hybrid_clock.cc:648] HybridClock initialized: now 1786091846684910 us; error 50 us; skew 500 ppm
Aug 07 08:37:26 dist-test-slave-h6n0 krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091846, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:26.687909 14672 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:26.688436 14672 webserver.cc:460] Webserver started at http://127.10.207.1:41093/ using document root <none> and password file <none>
I20260807 08:37:26.688679 14672 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:26.688750 14672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:26.688899 14672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:26.689947 14672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "0b14a289bfe64c9db8934efa05aee769"
format_stamp: "Formatted at 2026-08-07 08:37:26 on dist-test-slave-h6n0"
server_key: "39953c689f3bebff5057229c736d5ead"
server_key_iv: "7a7c986b3d9a0799aee5003a0ce0a710"
server_key_version: "encryptionkey@0"
I20260807 08:37:26.690322 14672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "0b14a289bfe64c9db8934efa05aee769"
format_stamp: "Formatted at 2026-08-07 08:37:26 on dist-test-slave-h6n0"
server_key: "39953c689f3bebff5057229c736d5ead"
server_key_iv: "7a7c986b3d9a0799aee5003a0ce0a710"
server_key_version: "encryptionkey@0"
I20260807 08:37:26.691895 14672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:26.692945 14696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:26.693204 14672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:26.693320 14672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "0b14a289bfe64c9db8934efa05aee769"
format_stamp: "Formatted at 2026-08-07 08:37:26 on dist-test-slave-h6n0"
server_key: "39953c689f3bebff5057229c736d5ead"
server_key_iv: "7a7c986b3d9a0799aee5003a0ce0a710"
server_key_version: "encryptionkey@0"
I20260807 08:37:26.693416 14672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:26.706341 14672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:26.708798 14672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:26.708968 14672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:26.709208 14672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:26.709544 14672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:26.709600 14672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:26.709636 14672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:26.709666 14672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:26.717458 14672 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:35759
I20260807 08:37:26.717563 14859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:35759 every 8 connection(s)
I20260807 08:37:26.717895 14672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:26.724781 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 14672
I20260807 08:37:26.724867 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:26.725116 11068 external_mini_cluster.cc:1428] Setting key 13bf1642b511c1d57a7d08b659477487
Aug 07 08:37:26 dist-test-slave-h6n0 krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091846, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:26.730428 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:26.718854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:59143 (local address 127.10.207.62:37215)
0807 08:37:26.718977 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:26.718979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:26.719243 (+ 264us) server_negotiation.cc:407] Connection header received
0807 08:37:26.719872 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:26.719873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:26.719887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:26.720232 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:26.721595 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:26.722158 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:26.723407 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:26.723646 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:26.723909 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:26.726776 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:26.726778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:26.726779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:26.726787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:26.728747 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:26.729318 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:26.729319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:26.729319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:26.729383 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:26.729697 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:26.729698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:26.729699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:26.729831 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:26.729895 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:26.730263 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 08:37:26.730344 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260807 08:37:26.730741 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:26.718834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:37215 (local address 127.10.207.1:59143)
0807 08:37:26.719054 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:26.719100 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:26.719742 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:26.720342 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:26.720346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:26.721063 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:26.721355 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:26.721361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:26.722274 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:26.722274 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:26.723305 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:26.723308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:26.723913 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:26.723914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:26.724032 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:26.724632 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:26.724641 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:26.726667 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:26.728858 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:26.728866 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:26.728869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:26.729225 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:26.729466 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:26.729467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:26.729467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:26.729612 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:26.729912 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:26.729918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:26.730107 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 08:37:26.730233 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 08:37:26.730425 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":123,"threads_started":1}
I20260807 08:37:26.731460 14860 heartbeater.cc:347] Connected to a master server at 127.10.207.62:37215
I20260807 08:37:26.731594 14860 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:26.731843 14860 heartbeater.cc:511] Master 127.10.207.62:37215 requested a full tablet report, sending...
I20260807 08:37:26.732489 14584 ts_manager.cc:194] Registered new tserver with Master: 0b14a289bfe64c9db8934efa05aee769 (127.10.207.1:35759)
I20260807 08:37:26.733139 14584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:59143
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
W20260807 08:37:27.554920 11068 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.207.2: real 0.829s user 0.002s sys 0.002s
I20260807 08:37:27.555335 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37215
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:27.643702 14871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:27.643898 14871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:27.643925 14871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:27.644984 14871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:27.645021 14871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:27.645046 14871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:27.646404 14871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:27.646665 14871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:27.646925 14871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:27.652104 14880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:27.652109 14876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:27.652211 14877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:27.652594 14871 server_base.cc:1034] running on GCE node
I20260807 08:37:27.652792 14871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:27.652982 14871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:27.654162 14871 hybrid_clock.cc:648] HybridClock initialized: now 1786091847654136 us; error 61 us; skew 500 ppm
Aug 07 08:37:27 dist-test-slave-h6n0 krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:27.661217 14871 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:27.661691 14871 webserver.cc:460] Webserver started at http://127.10.207.2:45279/ using document root <none> and password file <none>
I20260807 08:37:27.661875 14871 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:27.661918 14871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:27.662009 14871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:27.663049 14871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "87c13aa29ebb4458b32a8d467845617d"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h6n0"
server_key: "929ccf07d1102866c366e71fdbe0e9d1"
server_key_iv: "eb8969c14909dc9ef2804e965edd23b2"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.663363 14871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "87c13aa29ebb4458b32a8d467845617d"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h6n0"
server_key: "929ccf07d1102866c366e71fdbe0e9d1"
server_key_iv: "eb8969c14909dc9ef2804e965edd23b2"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.664937 14871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:27.665763 14889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.666049 14871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:27.666160 14871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "87c13aa29ebb4458b32a8d467845617d"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h6n0"
server_key: "929ccf07d1102866c366e71fdbe0e9d1"
server_key_iv: "eb8969c14909dc9ef2804e965edd23b2"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.666245 14871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:27.680264 14871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:27.682672 14871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:27.682840 14871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:27.683070 14871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:27.683385 14871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:27.683452 14871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.683492 14871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:27.683519 14871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.690732 14871 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:44303
I20260807 08:37:27.690788 15058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:44303 every 8 connection(s)
I20260807 08:37:27.691085 14871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:27.691448 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 14871
I20260807 08:37:27.691552 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:27.691766 11068 external_mini_cluster.cc:1428] Setting key b8b6e52dfb3a024ce94ccd35f1cac3fb
Aug 07 08:37:27 dist-test-slave-h6n0 krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
I20260807 08:37:27.703423 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.691961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:50983 (local address 127.10.207.62:37215)
0807 08:37:27.692126 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:27.692129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:27.692311 (+ 182us) server_negotiation.cc:407] Connection header received
0807 08:37:27.693047 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:27.693048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:27.693074 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:27.693163 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:27.694287 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.694960 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.696198 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.696592 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.696697 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.699773 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:27.699776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:27.699776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:27.699788 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:27.701772 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.702295 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.702296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.702297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.702377 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.702741 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.702743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.702743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.702888 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:27.702952 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:27.703245 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 08:37:27.703335 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260807 08:37:27.703619 15061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.691902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:37215 (local address 127.10.207.2:50983)
0807 08:37:27.692121 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.692181 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:27.692925 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:27.693257 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:27.693262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:27.693935 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:27.694173 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.694178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.695125 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.695126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.696084 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.696088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.696713 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.696714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.696839 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.697662 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:27.697674 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:27.699652 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:27.701889 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.701898 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.701900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.702197 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.702468 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.702469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.702470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.702647 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.702961 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:27.702966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:27.703094 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 08:37:27.703185 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 08:37:27.703400 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260807 08:37:27.704280 15059 heartbeater.cc:347] Connected to a master server at 127.10.207.62:37215
I20260807 08:37:27.704419 15059 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:27.704610 15059 heartbeater.cc:511] Master 127.10.207.62:37215 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
I20260807 08:37:27.705142 14584 ts_manager.cc:194] Registered new tserver with Master: 87c13aa29ebb4458b32a8d467845617d (127.10.207.2:44303)
I20260807 08:37:27.705629 14584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:50983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:27.722046 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37215
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:27.734340 14860 heartbeater.cc:503] Master 127.10.207.62:37215 was elected leader, sending a full tablet report...
W20260807 08:37:27.815270 15066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:27.815538 15066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:27.815577 15066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:27.816731 15066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:27.816807 15066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:27.816852 15066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:27.818367 15066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:27.818686 15066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:27.818951 15066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:27.825147 15075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:27.825155 15076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:27.825146 15080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:27.825636 15066 server_base.cc:1034] running on GCE node
I20260807 08:37:27.825889 15066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:27.826176 15066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:27.827351 15066 hybrid_clock.cc:648] HybridClock initialized: now 1786091847827333 us; error 59 us; skew 500 ppm
Aug 07 08:37:27 dist-test-slave-h6n0 krb5kdc[14517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:27.830210 15066 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:27.830766 15066 webserver.cc:460] Webserver started at http://127.10.207.3:44013/ using document root <none> and password file <none>
I20260807 08:37:27.831019 15066 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:27.831072 15066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:27.831224 15066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:27.832466 15066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "90227182784042e390e8fe370d025176"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h6n0"
server_key: "4c65b4c9f72de12bcf0d5908bb3c0821"
server_key_iv: "4ee1d6a28ae2f56d1d91310d7f3e00ce"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.832881 15066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "90227182784042e390e8fe370d025176"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h6n0"
server_key: "4c65b4c9f72de12bcf0d5908bb3c0821"
server_key_iv: "4ee1d6a28ae2f56d1d91310d7f3e00ce"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.834446 15066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:27.835465 15090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.835729 15066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:27.835852 15066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "90227182784042e390e8fe370d025176"
format_stamp: "Formatted at 2026-08-07 08:37:27 on dist-test-slave-h6n0"
server_key: "4c65b4c9f72de12bcf0d5908bb3c0821"
server_key_iv: "4ee1d6a28ae2f56d1d91310d7f3e00ce"
server_key_version: "encryptionkey@0"
I20260807 08:37:27.835937 15066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:27.870972 15066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:27.873468 15066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:27.873687 15066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:27.874014 15066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:27.874408 15066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:27.874469 15066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.874516 15066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:27.874565 15066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:27.882694 15066 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:35273
I20260807 08:37:27.882750 15254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:35273 every 8 connection(s)
I20260807 08:37:27.883159 15066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:27.888655 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 15066
I20260807 08:37:27.888896 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:27.889128 11068 external_mini_cluster.cc:1428] Setting key 664f9ee3dd07cb01e52773229116220b
Aug 07 08:37:27 dist-test-slave-h6n0 krb5kdc[14517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091847, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:27.894346 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.884109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:55173 (local address 127.10.207.62:37215)
0807 08:37:27.884192 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:27.884195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:27.884512 (+ 317us) server_negotiation.cc:407] Connection header received
0807 08:37:27.885173 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:27.885174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:27.885187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:27.885249 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:27.886370 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.887035 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.888129 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.888360 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.888449 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.891413 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:27.891415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:27.891416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:27.891423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:27.892810 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.893292 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.893293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.893293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.893358 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:27.893655 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:27.893656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:27.893657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:27.893792 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:27.893859 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:27.894165 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 08:37:27.894262 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:37:27.894616 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:27.884073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:37215 (local address 127.10.207.3:55173)
0807 08:37:27.884352 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:27.884396 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:27.885046 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:27.885338 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:27.885345 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:27.886008 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:27.886253 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.886258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.887146 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.887147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.888005 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:27.888007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:27.888460 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:27.888461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:27.888600 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:27.889264 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:27.889278 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:27.891290 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:27.892930 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.892936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.892939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.893191 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.893441 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:27.893442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:27.893442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:27.893554 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:27.893864 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:27.893869 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:27.894011 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 08:37:27.894104 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:37:27.894348 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":187,"threads_started":1}
I20260807 08:37:27.895155 15255 heartbeater.cc:347] Connected to a master server at 127.10.207.62:37215
I20260807 08:37:27.895308 15255 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:27.895614 15255 heartbeater.cc:511] Master 127.10.207.62:37215 requested a full tablet report, sending...
I20260807 08:37:27.896183 14584 ts_manager.cc:194] Registered new tserver with Master: 90227182784042e390e8fe370d025176 (127.10.207.3:35273)
I20260807 08:37:27.896613 14584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:55173
I20260807 08:37:27.901474 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:37:27.916687 15062 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:27.915465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.10.207.62:37215)
0807 08:37:27.915574 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:27.915577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:27.915717 (+ 140us) server_negotiation.cc:407] Connection header received
0807 08:37:27.915831 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:27.915833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:27.915845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:27.915937 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:27.916613 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48852: BlockingRecv error: recv got EOF from 127.0.0.1:48852 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:37:27.918202 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 14672
I20260807 08:37:27.925312 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 14871
I20260807 08:37:27.951479 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 15066
I20260807 08:37:27.964610 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 14537
2026-08-07T08:37:27Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3821 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:28 dist-test-slave-h6n0 krb5kdc[15273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:28 dist-test-slave-h6n0 krb5kdc[15273](info): set up 2 sockets
Aug 07 08:37:28 dist-test-slave-h6n0 krb5kdc[15273](info): commencing operation
krb5kdc: starting...
W20260807 08:37:29.230887 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:29 dist-test-slave-h6n0 krb5kdc[15273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:29.371451 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35419
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:35419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:29.459383 15293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:29.459616 15293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:29.459656 15293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:29.460754 15293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:29.460819 15293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:29.460846 15293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:29.460877 15293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:29.460902 15293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:29.462385 15293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:35419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:29.462677 15293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:29.462906 15293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:29.468236 15300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:29.468243 15303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:29.468425 15293 server_base.cc:1034] running on GCE node
W20260807 08:37:29.468243 15301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:29.468968 15293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:29.469367 15293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:29.470615 15293 hybrid_clock.cc:648] HybridClock initialized: now 1786091849470567 us; error 70 us; skew 500 ppm
Aug 07 08:37:29 dist-test-slave-h6n0 krb5kdc[15273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091849, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:29.473723 15293 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:29.474272 15293 webserver.cc:460] Webserver started at http://127.10.207.62:43275/ using document root <none> and password file <none>
I20260807 08:37:29.474634 15293 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:29.474713 15293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:29.474881 15293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:29.476085 15293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "651aa81ec6d746d3af17ddd8f414c5f0"
format_stamp: "Formatted at 2026-08-07 08:37:29 on dist-test-slave-h6n0"
server_key: "8adc3c0e217c36fc4992d432d93ed2db"
server_key_iv: "01c7ee8a7b22ac83d20f6be3f72f98cd"
server_key_version: "encryptionkey@0"
I20260807 08:37:29.476493 15293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "651aa81ec6d746d3af17ddd8f414c5f0"
format_stamp: "Formatted at 2026-08-07 08:37:29 on dist-test-slave-h6n0"
server_key: "8adc3c0e217c36fc4992d432d93ed2db"
server_key_iv: "01c7ee8a7b22ac83d20f6be3f72f98cd"
server_key_version: "encryptionkey@0"
I20260807 08:37:29.478101 15293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:29.479195 15312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:29.479442 15293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:29.479571 15293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "651aa81ec6d746d3af17ddd8f414c5f0"
format_stamp: "Formatted at 2026-08-07 08:37:29 on dist-test-slave-h6n0"
server_key: "8adc3c0e217c36fc4992d432d93ed2db"
server_key_iv: "01c7ee8a7b22ac83d20f6be3f72f98cd"
server_key_version: "encryptionkey@0"
I20260807 08:37:29.479678 15293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:29.501773 15293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:29.504590 15293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:29.504787 15293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:29.510212 15293 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:35419
I20260807 08:37:29.510251 15384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:35419 every 8 connection(s)
I20260807 08:37:29.510949 15293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:29.511888 15385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:29.514355 15385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0: Bootstrap starting.
I20260807 08:37:29.515023 15385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:29.515277 15385 log.cc:824] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:29.516113 15385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0: No bootstrap required, opened a new log
I20260807 08:37:29.517716 15385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35419 } }
I20260807 08:37:29.517859 15385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:29.517882 15385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 651aa81ec6d746d3af17ddd8f414c5f0, State: Initialized, Role: FOLLOWER
I20260807 08:37:29.518042 15385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35419 } }
I20260807 08:37:29.518143 15385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:29.518169 15385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:29.518241 15385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:29.518334 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 15293
I20260807 08:37:29.518452 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:29.518695 11068 external_mini_cluster.cc:1428] Setting key a0f616240b561cd663b8fe18f314f8f1
I20260807 08:37:29.519501 15385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35419 } }
I20260807 08:37:29.519631 15385 leader_election.cc:302] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 651aa81ec6d746d3af17ddd8f414c5f0; no voters:
I20260807 08:37:29.519840 15385 leader_election.cc:288] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:29.519878 15393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:29.520018 15393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [term 1 LEADER]: Becoming Leader. State: Replica: 651aa81ec6d746d3af17ddd8f414c5f0, State: Running, Role: LEADER
I20260807 08:37:29.520143 15385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:29.520193 15393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35419 } }
I20260807 08:37:29.520529 15393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35419 } } }
I20260807 08:37:29.520614 15393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:29.520759 15393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 651aa81ec6d746d3af17ddd8f414c5f0. Latest consensus state: current_term: 1 leader_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651aa81ec6d746d3af17ddd8f414c5f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35419 } } }
I20260807 08:37:29.520841 15393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0 [sys.catalog]: This master's current role is: LEADER
W20260807 08:37:29.522322 15415 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:29.522401 15415 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:29.522462 15402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:29.522683 15402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:29.524780 15402 catalog_manager.cc:1347] Generated new cluster ID: 03e734debba74857b0811b86b668c64e
I20260807 08:37:29.524827 15402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:37:29 dist-test-slave-h6n0 krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:29.529963 15402 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:29.530733 15402 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:29.533689 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:29.519340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.10.207.62:35419)
0807 08:37:29.519551 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:29.519561 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:29.519597 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:37:29.519778 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:29.519784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:29.519838 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:29.520099 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:29.521074 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:29.522492 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:29.523660 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:29.524314 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:29.529438 (+ 5124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:29.529558 (+ 120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:29.529562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:29.529562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:29.529574 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:29.531847 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:29.532324 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:29.532328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:29.532328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:29.532445 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:29.532689 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:29.532690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:29.532690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:29.532972 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:29.533131 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:29.533263 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:37:29.533426 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260807 08:37:29.534844 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:37:29.543355 15402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 651aa81ec6d746d3af17ddd8f414c5f0: Generated new TSK 0
I20260807 08:37:29.543565 15402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:37:29.558001 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:29.553514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34990 (local address 127.10.207.62:35419)
0807 08:37:29.553591 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:29.553594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:29.553739 (+ 145us) server_negotiation.cc:407] Connection header received
0807 08:37:29.553782 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:29.553783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:29.553796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:29.553881 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:29.554296 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:29.555234 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:29.556075 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:29.556413 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:29.556503 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:29.557382 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:29.557383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:29.557384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:29.557648 (+ 264us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:29.557671 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:29.557679 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:29.557752 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:29.557877 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 08:37:29.557908 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:37:29.561296 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 15293
2026-08-07T08:37:29Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1599 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:29 dist-test-slave-h6n0 krb5kdc[15432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:29 dist-test-slave-h6n0 krb5kdc[15432](info): set up 2 sockets
Aug 07 08:37:29 dist-test-slave-h6n0 krb5kdc[15432](info): commencing operation
krb5kdc: starting...
W20260807 08:37:30.844384 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:30 dist-test-slave-h6n0 krb5kdc[15432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:30.981739 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:38057
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:38057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:31.076617 15452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:31.076903 15452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:31.076942 15452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:31.078126 15452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:31.078195 15452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:31.078220 15452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:31.078243 15452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:31.078270 15452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:31.079903 15452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:38057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:38057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:31.080225 15452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:31.080492 15452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:31.086395 15461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.086342 15459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.086413 15458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:31.086742 15452 server_base.cc:1034] running on GCE node
I20260807 08:37:31.087028 15452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:31.087399 15452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:31.088616 15452 hybrid_clock.cc:648] HybridClock initialized: now 1786091851088579 us; error 51 us; skew 500 ppm
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:31.091703 15452 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:31.092211 15452 webserver.cc:460] Webserver started at http://127.10.207.62:43111/ using document root <none> and password file <none>
I20260807 08:37:31.092501 15452 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:31.092574 15452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:31.092813 15452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:31.093991 15452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "50158278cba44405b42b3e93c3d2a415"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "ab5eb46b0fc4af353cc7256a3acd2610"
server_key_iv: "d0989f7ce98526b301374165f685c4bf"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.094395 15452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "50158278cba44405b42b3e93c3d2a415"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "ab5eb46b0fc4af353cc7256a3acd2610"
server_key_iv: "d0989f7ce98526b301374165f685c4bf"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.096030 15452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:31.097230 15471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.097506 15452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:31.097622 15452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "50158278cba44405b42b3e93c3d2a415"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "ab5eb46b0fc4af353cc7256a3acd2610"
server_key_iv: "d0989f7ce98526b301374165f685c4bf"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.097719 15452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:31.117393 15452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:31.119805 15452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:31.119987 15452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:31.124554 15452 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:38057
I20260807 08:37:31.124632 15540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:38057 every 8 connection(s)
I20260807 08:37:31.125020 15452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:31.126418 15541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:31.128244 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 15452
I20260807 08:37:31.128373 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:31.128572 11068 external_mini_cluster.cc:1428] Setting key 81749e4125ee851f16ed0f4010e70c3a
I20260807 08:37:31.129886 15541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415: Bootstrap starting.
I20260807 08:37:31.130837 15541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:31.131235 15541 log.cc:824] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:31.132077 15541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415: No bootstrap required, opened a new log
I20260807 08:37:31.134903 15541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50158278cba44405b42b3e93c3d2a415" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38057 } }
I20260807 08:37:31.135121 15541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:31.135165 15541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50158278cba44405b42b3e93c3d2a415, State: Initialized, Role: FOLLOWER
I20260807 08:37:31.135332 15541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50158278cba44405b42b3e93c3d2a415" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38057 } }
I20260807 08:37:31.135453 15541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:31.135493 15541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:31.135562 15541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:31.136394 15541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50158278cba44405b42b3e93c3d2a415" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38057 } }
I20260807 08:37:31.136499 15541 leader_election.cc:302] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50158278cba44405b42b3e93c3d2a415; no voters:
I20260807 08:37:31.136699 15541 leader_election.cc:288] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:31.136721 15549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 1 FOLLOWER]: Leader election won for term 1
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:31.136874 15549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [term 1 LEADER]: Becoming Leader. State: Replica: 50158278cba44405b42b3e93c3d2a415, State: Running, Role: LEADER
I20260807 08:37:31.136960 15549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50158278cba44405b42b3e93c3d2a415" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38057 } }
I20260807 08:37:31.137022 15541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:31.137343 15549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50158278cba44405b42b3e93c3d2a415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50158278cba44405b42b3e93c3d2a415" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38057 } } }
I20260807 08:37:31.137490 15549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:31.137715 15549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50158278cba44405b42b3e93c3d2a415. Latest consensus state: current_term: 1 leader_uuid: "50158278cba44405b42b3e93c3d2a415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50158278cba44405b42b3e93c3d2a415" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38057 } } }
I20260807 08:37:31.137859 15549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:31.138266 15556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:31.139294 15556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:31.141106 15556 catalog_manager.cc:1347] Generated new cluster ID: 084057493e9146dc8135f8bb93496611
I20260807 08:37:31.141165 15556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:31.142748 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.129198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.10.207.62:38057)
0807 08:37:31.129406 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:31.129414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:31.129442 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:37:31.130062 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:31.130070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:31.130119 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:31.130242 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:31.131257 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.132289 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.133432 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.133918 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.134017 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.137671 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:31.137675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:31.137676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:31.137689 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:31.140382 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.140858 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.140862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.140862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.141221 (+ 359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.141536 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.141537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.141538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.141862 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:31.142052 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:31.142234 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 08:37:31.142507 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260807 08:37:31.148756 15556 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:31.149516 15556 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:31.153141 15556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50158278cba44405b42b3e93c3d2a415: Generated new TSK 0
I20260807 08:37:31.153308 15556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:31.187762 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38057
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:31.300233 15582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:31.300490 15582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:31.300529 15582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:31.301899 15582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:31.301967 15582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:31.302016 15582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:31.303818 15582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:31.304121 15582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:31.304394 15582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:31.310140 15589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:31.310271 15582 server_base.cc:1034] running on GCE node
W20260807 08:37:31.310105 15588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.310118 15592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:31.310652 15582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:31.311009 15582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:31.312170 15582 hybrid_clock.cc:648] HybridClock initialized: now 1786091851312152 us; error 37 us; skew 500 ppm
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:31.315132 15582 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:31.315655 15582 webserver.cc:460] Webserver started at http://127.10.207.1:34569/ using document root <none> and password file <none>
I20260807 08:37:31.315898 15582 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:31.315944 15582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:31.316049 15582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:31.317214 15582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "ecfc09f1d805437fa6285463a8ae6c62"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "6e3ba47847add7a59e656d346aa5b095"
server_key_iv: "d634cf878ee40a10af30c3857c945f82"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.317642 15582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "ecfc09f1d805437fa6285463a8ae6c62"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "6e3ba47847add7a59e656d346aa5b095"
server_key_iv: "d634cf878ee40a10af30c3857c945f82"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.319259 15582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:31.320087 15602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.320355 15582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:31.320482 15582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "ecfc09f1d805437fa6285463a8ae6c62"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "6e3ba47847add7a59e656d346aa5b095"
server_key_iv: "d634cf878ee40a10af30c3857c945f82"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.320585 15582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:31.328776 15582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:31.330999 15582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:31.331180 15582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:31.331449 15582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:31.331807 15582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:31.331866 15582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.331899 15582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:31.331929 15582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.340137 15582 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:39451
I20260807 08:37:31.340191 15762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:39451 every 8 connection(s)
I20260807 08:37:31.340548 15582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:31.345633 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 15582
I20260807 08:37:31.345736 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:31.345901 11068 external_mini_cluster.cc:1428] Setting key 44118e526d87fd8fb44f471e408f9abf
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:31.351226 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.341329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:41853 (local address 127.10.207.62:38057)
0807 08:37:31.341417 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:31.341420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:31.341739 (+ 319us) server_negotiation.cc:407] Connection header received
0807 08:37:31.342275 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:31.342276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:31.342295 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:31.342361 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:31.343433 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.343992 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.345085 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.345315 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.345392 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.347655 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:31.347657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:31.347658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:31.347666 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:31.349463 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.350029 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.350031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.350031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.350097 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.350400 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.350401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.350401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.350590 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:31.350671 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:31.350970 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 08:37:31.351085 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:37:31.351267 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.341320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38057 (local address 127.10.207.1:41853)
0807 08:37:31.341562 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:31.341624 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:31.342172 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:31.342445 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:31.342448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:31.343112 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:31.343315 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:31.343321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.344118 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.344119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:31.344959 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:31.344961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.345421 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.345422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:31.345536 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.346083 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:31.346091 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:31.347548 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:31.349607 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:31.349620 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:31.349623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:31.349935 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:31.350183 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:31.350184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:31.350184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:31.350319 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:31.350662 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:31.350669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:31.350815 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 08:37:31.350917 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 08:37:31.351068 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":135,"threads_started":1}
I20260807 08:37:31.351909 15763 heartbeater.cc:347] Connected to a master server at 127.10.207.62:38057
I20260807 08:37:31.352018 15763 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:31.352283 15763 heartbeater.cc:511] Master 127.10.207.62:38057 requested a full tablet report, sending...
I20260807 08:37:31.352946 15495 ts_manager.cc:194] Registered new tserver with Master: ecfc09f1d805437fa6285463a8ae6c62 (127.10.207.1:39451)
I20260807 08:37:31.353653 15495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:41853
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:31.376811 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38057
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:31.462263 15769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:31.462455 15769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:31.462477 15769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:31.463550 15769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:31.463591 15769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:31.463617 15769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:31.464985 15769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:31.465167 15769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:31.465341 15769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:31.470621 15782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.470715 15778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.470865 15777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:31.470852 15769 server_base.cc:1034] running on GCE node
I20260807 08:37:31.471134 15769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:31.471338 15769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:31.472484 15769 hybrid_clock.cc:648] HybridClock initialized: now 1786091851472467 us; error 45 us; skew 500 ppm
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:31.475234 15769 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:31.475739 15769 webserver.cc:460] Webserver started at http://127.10.207.2:42101/ using document root <none> and password file <none>
I20260807 08:37:31.475996 15769 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:31.476065 15769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:31.476225 15769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:31.477258 15769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "9aa8ac5593e34bff8bb7a1e06b67955b"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "8ba754156801824ed2911a1a78ac3945"
server_key_iv: "c0e5da66bb50fd6cf892f94ea74533ec"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.477630 15769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "9aa8ac5593e34bff8bb7a1e06b67955b"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "8ba754156801824ed2911a1a78ac3945"
server_key_iv: "c0e5da66bb50fd6cf892f94ea74533ec"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.479146 15769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:31.480024 15791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.480293 15769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:31.480418 15769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "9aa8ac5593e34bff8bb7a1e06b67955b"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "8ba754156801824ed2911a1a78ac3945"
server_key_iv: "c0e5da66bb50fd6cf892f94ea74533ec"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.480506 15769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:31.501614 15769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:31.504300 15769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:31.504480 15769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:31.504724 15769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:31.505059 15769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:31.505118 15769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.505164 15769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:31.505191 15769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.512307 15769 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:44025
I20260807 08:37:31.512440 15947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:44025 every 8 connection(s)
I20260807 08:37:31.512755 15769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:31.522841 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 15769
I20260807 08:37:31.523000 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:31.523226 11068 external_mini_cluster.cc:1428] Setting key a18d7e3f422ba864f8bb30305286136f
I20260807 08:37:31.524730 15951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.513617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38057 (local address 127.10.207.2:46193)
0807 08:37:31.513783 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:31.513824 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:31.514306 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:31.514675 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:31.514679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:31.515227 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:31.515432 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:31.515437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.516282 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.516283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:31.517363 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:31.517369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.517827 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.517828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:31.517967 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.518765 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:31.518775 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:31.520562 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:31.522403 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:31.522409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:31.522411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:31.522837 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:31.523161 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:31.523163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:31.523164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:31.523382 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:31.523965 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:31.523974 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:31.524185 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 08:37:31.524315 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 08:37:31.524487 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260807 08:37:31.524761 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.513819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:46193 (local address 127.10.207.62:38057)
0807 08:37:31.513884 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:31.513886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:31.513909 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:37:31.514433 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:31.514435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:31.514447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:31.514602 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:31.515557 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.516133 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.517475 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.517700 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.517784 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.520682 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:31.520685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:31.520685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:31.520694 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:31.522299 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.522954 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.522956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.522956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.523055 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.523549 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.523551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.523552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.523774 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:31.523976 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:31.524400 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 08:37:31.524524 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:37:31.525465 15948 heartbeater.cc:347] Connected to a master server at 127.10.207.62:38057
I20260807 08:37:31.525584 15948 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:31.525795 15948 heartbeater.cc:511] Master 127.10.207.62:38057 requested a full tablet report, sending...
I20260807 08:37:31.526365 15495 ts_manager.cc:194] Registered new tserver with Master: 9aa8ac5593e34bff8bb7a1e06b67955b (127.10.207.2:44025)
I20260807 08:37:31.526983 15495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:46193
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:31.553654 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38057
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:31.648187 15956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:31.648452 15956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:31.648483 15956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:31.649703 15956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:31.649742 15956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:31.649770 15956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:31.651422 15956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:31.651669 15956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:31.651898 15956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:31.657864 15967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.657857 15964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:31.657912 15965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:31.658098 15956 server_base.cc:1034] running on GCE node
I20260807 08:37:31.658492 15956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:31.658900 15956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:31.660058 15956 hybrid_clock.cc:648] HybridClock initialized: now 1786091851660042 us; error 72 us; skew 500 ppm
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:31.662952 15956 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:31.663440 15956 webserver.cc:460] Webserver started at http://127.10.207.3:35557/ using document root <none> and password file <none>
I20260807 08:37:31.663712 15956 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:31.663765 15956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:31.663946 15956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:31.665113 15956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "4b868d6bab684d23b3ae3a1b06ce9605"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "bde4655d802afa99300997065b969ec8"
server_key_iv: "4984f3a1f92da83314a406f799258b3a"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.665495 15956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "4b868d6bab684d23b3ae3a1b06ce9605"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "bde4655d802afa99300997065b969ec8"
server_key_iv: "4984f3a1f92da83314a406f799258b3a"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.667099 15956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:31.667995 15976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.668280 15956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:31.668406 15956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "4b868d6bab684d23b3ae3a1b06ce9605"
format_stamp: "Formatted at 2026-08-07 08:37:31 on dist-test-slave-h6n0"
server_key: "bde4655d802afa99300997065b969ec8"
server_key_iv: "4984f3a1f92da83314a406f799258b3a"
server_key_version: "encryptionkey@0"
I20260807 08:37:31.668485 15956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:31.683550 15956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:31.685563 15956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:31.685732 15956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:31.685958 15956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:31.686303 15956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:31.686371 15956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.686414 15956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:31.686443 15956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:31.693554 15956 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:40279
I20260807 08:37:31.693686 16137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:40279 every 8 connection(s)
I20260807 08:37:31.693974 15956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:31.700537 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 15956
I20260807 08:37:31.700654 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:31.700904 11068 external_mini_cluster.cc:1428] Setting key 97ce4f77aa00d0b31a23bd2c71bcb4e2
I20260807 08:37:31.704968 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.694877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:45395 (local address 127.10.207.62:38057)
0807 08:37:31.694927 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:31.694929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:31.695226 (+ 297us) server_negotiation.cc:407] Connection header received
0807 08:37:31.695721 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:31.695722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:31.695736 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:31.695802 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:31.696711 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.697260 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.698236 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.698438 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.698497 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.701320 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:31.701322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:31.701322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:31.701329 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:31.703165 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.703636 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.703637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.703638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.703705 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.704063 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.704064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.704065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.704248 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:31.704323 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:31.704799 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 08:37:31.704893 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:37:31.705062 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.694836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38057 (local address 127.10.207.3:45395)
0807 08:37:31.695040 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:31.695101 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:31.695605 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:31.695857 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:31.695860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:31.696419 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:31.696602 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:31.696607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.697351 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.697351 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:31.698144 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:31.698146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.698520 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.698521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:31.698680 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.699255 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:31.699264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:31.701203 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:31.703287 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:31.703293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:31.703295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:31.703517 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:31.703816 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:31.703817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:31.703817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:31.703946 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:31.704359 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:31.704365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:31.704478 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:37:31.704708 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 08:37:31.704867 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260807 08:37:31.705869 16138 heartbeater.cc:347] Connected to a master server at 127.10.207.62:38057
I20260807 08:37:31.706032 16138 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:31.706293 16138 heartbeater.cc:511] Master 127.10.207.62:38057 requested a full tablet report, sending...
I20260807 08:37:31.707015 15495 ts_manager.cc:194] Registered new tserver with Master: 4b868d6bab684d23b3ae3a1b06ce9605 (127.10.207.3:40279)
I20260807 08:37:31.707412 15495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:45395
I20260807 08:37:31.713111 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[15432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091851, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:31.740852 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:31.732397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.10.207.62:38057)
0807 08:37:31.732501 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:31.732504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:31.732532 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:37:31.732572 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:31.732573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:31.732584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:31.732670 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:31.733327 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.733871 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.734730 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:31.734935 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:31.735022 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:31.737781 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:31.737783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:31.737783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:31.737791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:31.739336 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.739885 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.739886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.739886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.739949 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:31.740239 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:31.740239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:31.740240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:31.740434 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:31.740528 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:31.740694 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:37:31.740772 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 08:37:31.741179 15495 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36844
I20260807 08:37:31.741736 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 15582
I20260807 08:37:31.765916 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 15769
I20260807 08:37:31.774435 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 15956
I20260807 08:37:31.786439 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 15452
2026-08-07T08:37:31Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2223 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[16158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[16158](info): set up 2 sockets
Aug 07 08:37:31 dist-test-slave-h6n0 krb5kdc[16158](info): commencing operation
krb5kdc: starting...
W20260807 08:37:33.060549 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:33.201225 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:36541
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:34499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:36541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:33.289651 16176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:33.289849 16176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:33.289872 16176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:33.290943 16176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:33.290989 16176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:33.291004 16176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:33.291018 16176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:33.291029 16176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:33.292459 16176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:34499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:36541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:36541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:33.292647 16176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:33.292809 16176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:33.298197 16186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.298290 16176 server_base.cc:1034] running on GCE node
W20260807 08:37:33.298334 16185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.298193 16189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.298698 16176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:33.299007 16176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:33.300185 16176 hybrid_clock.cc:648] HybridClock initialized: now 1786091853300168 us; error 40 us; skew 500 ppm
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:33.307545 16176 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:33.307966 16176 webserver.cc:460] Webserver started at http://127.10.207.62:44021/ using document root <none> and password file <none>
I20260807 08:37:33.308223 16176 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:33.308295 16176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:33.308460 16176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:33.309582 16176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "7aecc766536e4c65a10dea52b6b066e0"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "e9fd434b4912826a61c8ccca2fd443a6"
server_key_iv: "cdefd883facf202e140ef3b6daafded8"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.309983 16176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "7aecc766536e4c65a10dea52b6b066e0"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "e9fd434b4912826a61c8ccca2fd443a6"
server_key_iv: "cdefd883facf202e140ef3b6daafded8"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.311537 16176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:33.312381 16201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.312638 16176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:33.312762 16176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "7aecc766536e4c65a10dea52b6b066e0"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "e9fd434b4912826a61c8ccca2fd443a6"
server_key_iv: "cdefd883facf202e140ef3b6daafded8"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.312839 16176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:33.327853 16176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.329847 16176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.329963 16176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.334586 16176 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:36541
I20260807 08:37:33.334663 16279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:36541 every 8 connection(s)
I20260807 08:37:33.335176 16176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:33.336021 16280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:33.338654 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 16176
I20260807 08:37:33.338757 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:33.338887 16280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0: Bootstrap starting.
I20260807 08:37:33.338958 11068 external_mini_cluster.cc:1428] Setting key c3d769616338a8404be2e6e005fe698c
I20260807 08:37:33.339824 16280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:33.340224 16280 log.cc:824] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:33.341102 16280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0: No bootstrap required, opened a new log
I20260807 08:37:33.343739 16280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aecc766536e4c65a10dea52b6b066e0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36541 } }
I20260807 08:37:33.343952 16280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:33.343989 16280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aecc766536e4c65a10dea52b6b066e0, State: Initialized, Role: FOLLOWER
I20260807 08:37:33.344146 16280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aecc766536e4c65a10dea52b6b066e0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36541 } }
I20260807 08:37:33.344239 16280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:33.344278 16280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:33.344327 16280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:33.345216 16280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aecc766536e4c65a10dea52b6b066e0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36541 } }
I20260807 08:37:33.345343 16280 leader_election.cc:302] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7aecc766536e4c65a10dea52b6b066e0; no voters:
I20260807 08:37:33.345561 16280 leader_election.cc:288] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:33.345598 16287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:33.345768 16287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [term 1 LEADER]: Becoming Leader. State: Replica: 7aecc766536e4c65a10dea52b6b066e0, State: Running, Role: LEADER
I20260807 08:37:33.345878 16287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aecc766536e4c65a10dea52b6b066e0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36541 } }
I20260807 08:37:33.345881 16280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:33.346400 16289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7aecc766536e4c65a10dea52b6b066e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aecc766536e4c65a10dea52b6b066e0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36541 } } }
I20260807 08:37:33.346504 16289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:33.346410 16290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7aecc766536e4c65a10dea52b6b066e0. Latest consensus state: current_term: 1 leader_uuid: "7aecc766536e4c65a10dea52b6b066e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aecc766536e4c65a10dea52b6b066e0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36541 } } }
I20260807 08:37:33.346707 16290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:33.346901 16294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:33.347565 16294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:33.349704 16294 catalog_manager.cc:1347] Generated new cluster ID: fb6706bd2fc6418bb3b20b625d24e05d
I20260807 08:37:33.349768 16294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:33.351914 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.339589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.10.207.62:36541)
0807 08:37:33.339837 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.339848 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.339888 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:37:33.340585 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.340593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.340646 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.340833 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.341493 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.343029 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.343828 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.344469 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.344593 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.347895 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.347900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.347900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.347921 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.350051 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.350407 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.350410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.350410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.350505 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.350737 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.350737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.350738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.351036 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.351155 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.351561 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.351691 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
I20260807 08:37:33.357478 16294 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:33.358294 16294 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:33.366963 16294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7aecc766536e4c65a10dea52b6b066e0: Generated new TSK 0
I20260807 08:37:33.367184 16294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:33.401320 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36541
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:34499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:33.486325 16313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:33.486589 16313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:33.486661 16313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:33.487845 16313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:33.487912 16313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:33.487959 16313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:33.489482 16313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:34499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:33.489739 16313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:33.489977 16313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:33.495258 16323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.495271 16318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.495258 16320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.495667 16313 server_base.cc:1034] running on GCE node
I20260807 08:37:33.495887 16313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:33.496086 16313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:33.497258 16313 hybrid_clock.cc:648] HybridClock initialized: now 1786091853497236 us; error 66 us; skew 500 ppm
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:33.500497 16313 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:33.501197 16313 webserver.cc:460] Webserver started at http://127.10.207.1:42901/ using document root <none> and password file <none>
I20260807 08:37:33.501494 16313 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:33.501549 16313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:33.501711 16313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:33.503083 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "c7a0f17490eb437fa9126e28b1ff8c5e"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "ba25731687c6290d14d3f643a834f058"
server_key_iv: "93db5fd9c2dbdeced720d334a50a6113"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.503518 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "c7a0f17490eb437fa9126e28b1ff8c5e"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "ba25731687c6290d14d3f643a834f058"
server_key_iv: "93db5fd9c2dbdeced720d334a50a6113"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.505285 16313 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:37:33.506464 16335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.506803 16313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:33.506963 16313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "c7a0f17490eb437fa9126e28b1ff8c5e"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "ba25731687c6290d14d3f643a834f058"
server_key_iv: "93db5fd9c2dbdeced720d334a50a6113"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.507078 16313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:33.519637 16313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.522591 16313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.522890 16313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.523258 16313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:33.523684 16313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:33.523743 16313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.523891 16313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:33.523963 16313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.531665 16313 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:37507
I20260807 08:37:33.531728 16503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:37507 every 8 connection(s)
I20260807 08:37:33.532061 16313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:33.537919 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 16313
I20260807 08:37:33.538026 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:33.538244 11068 external_mini_cluster.cc:1428] Setting key 900f593cadec03273ef9dc69821eda72
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:33.543494 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.532962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:53753 (local address 127.10.207.62:36541)
0807 08:37:33.533042 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.533044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.533238 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:37:33.534246 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.534247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.534260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.534304 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.535029 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.535550 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.536555 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.536771 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.536851 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.539691 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.539693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.539693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.539701 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.541385 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.542031 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.542033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.542034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.542127 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.542478 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.542480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.542480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.542786 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.542875 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.543218 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.543346 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:37:33.543526 16508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.532857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:36541 (local address 127.10.207.1:53753)
0807 08:37:33.533082 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.533133 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.534131 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.534372 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.534376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.534723 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:33.534922 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.534927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.535654 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.535655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.536459 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.536461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.536860 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.536861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.536984 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.537564 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:33.537573 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:33.539580 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:33.541497 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.541507 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.541510 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.541900 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.542214 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.542215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.542215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.542376 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.542896 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:33.542903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:33.543056 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.543169 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.543329 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":96,"threads_started":1}
I20260807 08:37:33.544200 16506 heartbeater.cc:347] Connected to a master server at 127.10.207.62:36541
I20260807 08:37:33.544324 16506 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:33.544551 16506 heartbeater.cc:511] Master 127.10.207.62:36541 requested a full tablet report, sending...
I20260807 08:37:33.545336 16223 ts_manager.cc:194] Registered new tserver with Master: c7a0f17490eb437fa9126e28b1ff8c5e (127.10.207.1:37507)
I20260807 08:37:33.546126 16223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:53753
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:33.570371 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36541
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:34499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:33.654560 16512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:33.654779 16512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:33.654801 16512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:33.655877 16512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:33.655927 16512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:33.655953 16512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:33.657507 16512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:34499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:33.657753 16512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:33.658020 16512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:33.663748 16527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.663750 16522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.663748 16521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.664089 16512 server_base.cc:1034] running on GCE node
I20260807 08:37:33.664343 16512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:33.664680 16512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:33.665860 16512 hybrid_clock.cc:648] HybridClock initialized: now 1786091853665855 us; error 42 us; skew 500 ppm
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:33.669086 16512 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:33.669639 16512 webserver.cc:460] Webserver started at http://127.10.207.2:43281/ using document root <none> and password file <none>
I20260807 08:37:33.669939 16512 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:33.670014 16512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:33.670185 16512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:33.671460 16512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "69bddc175ff64b1c8cd06d4b41121af2"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "6b26197547f35dc890db760235ba4e88"
server_key_iv: "76366713620fe3295a5f7ecb9dd7e96a"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.671865 16512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "69bddc175ff64b1c8cd06d4b41121af2"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "6b26197547f35dc890db760235ba4e88"
server_key_iv: "76366713620fe3295a5f7ecb9dd7e96a"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.673456 16512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:37:33.674487 16541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.674916 16512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:33.675081 16512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "69bddc175ff64b1c8cd06d4b41121af2"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "6b26197547f35dc890db760235ba4e88"
server_key_iv: "76366713620fe3295a5f7ecb9dd7e96a"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.675179 16512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:33.686023 16512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.688445 16512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.688732 16512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.689035 16512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:33.689436 16512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:33.689496 16512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.689536 16512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:33.689584 16512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.697666 16512 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:38663
I20260807 08:37:33.697773 16700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:38663 every 8 connection(s)
I20260807 08:37:33.698167 16512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:33.707198 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 16512
I20260807 08:37:33.707341 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:33.707554 11068 external_mini_cluster.cc:1428] Setting key 410c335f6dd977e2baf15c281f9064a2
I20260807 08:37:33.711324 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.699218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:42987 (local address 127.10.207.62:36541)
0807 08:37:33.699448 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.699451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.699820 (+ 369us) server_negotiation.cc:407] Connection header received
0807 08:37:33.701315 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.701317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.701334 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.701415 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.702285 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.703089 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.704465 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.704669 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.704751 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.707721 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.707723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.707724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.707740 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.709261 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.709897 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.709899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.709899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.709974 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.710279 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.710280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.710281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.710412 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.710499 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.711040 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.711181 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:37:33.711747 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.699165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:36541 (local address 127.10.207.2:42987)
0807 08:37:33.699496 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.699646 (+ 150us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.701074 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.701524 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.701529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.701883 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:33.702161 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.702166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.703378 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.703380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.704343 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.704345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.704856 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.704862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.705096 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.705764 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:33.705774 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:33.707558 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:33.709385 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.709393 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.709395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.709748 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.710061 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.710061 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.710062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.710190 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.710489 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:33.710497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:33.710683 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.710951 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.711325 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":108,"threads_started":1}
I20260807 08:37:33.712694 16701 heartbeater.cc:347] Connected to a master server at 127.10.207.62:36541
I20260807 08:37:33.712842 16701 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:33.713074 16701 heartbeater.cc:511] Master 127.10.207.62:36541 requested a full tablet report, sending...
I20260807 08:37:33.713551 16223 ts_manager.cc:194] Registered new tserver with Master: 69bddc175ff64b1c8cd06d4b41121af2 (127.10.207.2:38663)
I20260807 08:37:33.714078 16223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:42987
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:33.737654 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36541
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:34499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:33.823372 16707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:33.823588 16707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:33.823611 16707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:33.824733 16707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:33.824780 16707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:33.824806 16707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:33.826558 16707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:34499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:33.826855 16707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:33.827111 16707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:33.832787 16716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.832911 16713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:33.832803 16712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:33.833544 16707 server_base.cc:1034] running on GCE node
I20260807 08:37:33.833779 16707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:33.834028 16707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:33.835228 16707 hybrid_clock.cc:648] HybridClock initialized: now 1786091853835212 us; error 44 us; skew 500 ppm
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:33.838385 16707 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:33.838969 16707 webserver.cc:460] Webserver started at http://127.10.207.3:34303/ using document root <none> and password file <none>
I20260807 08:37:33.839255 16707 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:33.839336 16707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:33.839526 16707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:33.840775 16707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "bd5469dc9c354d27acfa434364242b90"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "0829c4cb73cecebc3cef690e31663012"
server_key_iv: "6b991a8e7e05bf7d992de233bf4a816f"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.841202 16707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "bd5469dc9c354d27acfa434364242b90"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "0829c4cb73cecebc3cef690e31663012"
server_key_iv: "6b991a8e7e05bf7d992de233bf4a816f"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.842932 16707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:33.843950 16726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.844266 16707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:33.844405 16707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "bd5469dc9c354d27acfa434364242b90"
format_stamp: "Formatted at 2026-08-07 08:37:33 on dist-test-slave-h6n0"
server_key: "0829c4cb73cecebc3cef690e31663012"
server_key_iv: "6b991a8e7e05bf7d992de233bf4a816f"
server_key_version: "encryptionkey@0"
I20260807 08:37:33.844492 16707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:33.855199 16707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:33.857805 16707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:33.857987 16707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:33.858283 16707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:33.858739 16707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:33.858798 16707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.858846 16707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:33.858873 16707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:33.866740 16707 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34217
I20260807 08:37:33.866902 16888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34217 every 8 connection(s)
I20260807 08:37:33.867395 16707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:33.874248 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 16707
I20260807 08:37:33.874341 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:33.874608 11068 external_mini_cluster.cc:1428] Setting key 2203eee159e4e49616c543241b4c1a38
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:33.878655 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.868406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:42605 (local address 127.10.207.62:36541)
0807 08:37:33.868499 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.868502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.868744 (+ 242us) server_negotiation.cc:407] Connection header received
0807 08:37:33.869899 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.869900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.869914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.869981 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.870782 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.871345 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.872420 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.872633 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.872724 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.875531 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.875533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.875534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.875541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.877043 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.877513 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.877514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.877515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.877584 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.877851 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.877852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.877852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.878005 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.878080 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.878433 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.878519 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:37:33.878886 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.868361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:36541 (local address 127.10.207.3:42605)
0807 08:37:33.868580 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:33.868635 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:33.869764 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:33.870064 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:33.870069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:33.870376 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:33.870687 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.870692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.871453 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.871454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.872326 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:33.872328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.872734 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.872736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:33.872885 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.873505 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:33.873515 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:33.875414 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:33.877148 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.877156 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.877159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.877415 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.877661 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:33.877662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:33.877663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:33.877781 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:33.878090 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:33.878098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:33.878289 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 08:37:33.878399 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 08:37:33.878616 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260807 08:37:33.879487 16889 heartbeater.cc:347] Connected to a master server at 127.10.207.62:36541
I20260807 08:37:33.879590 16889 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:33.879770 16889 heartbeater.cc:511] Master 127.10.207.62:36541 requested a full tablet report, sending...
I20260807 08:37:33.880255 16223 ts_manager.cc:194] Registered new tserver with Master: bd5469dc9c354d27acfa434364242b90 (127.10.207.3:34217)
I20260807 08:37:33.880712 16223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:42605
I20260807 08:37:33.886505 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:33.915329 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.906160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.10.207.1:37507)
0807 08:37:33.906491 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.906497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.906508 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:33.906600 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.906603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.906621 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.906712 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.907406 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.908245 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.909051 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.909450 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.909543 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.912309 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.912319 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.912319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.912328 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.913962 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.914360 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.914364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.914364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.914430 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.914677 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.914678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.914678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.914835 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.914999 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.915053 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.915186 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":197,"threads_started":1}
Aug 07 08:37:33 dist-test-slave-h6n0 krb5kdc[16158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:33.924703 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:33.917122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.10.207.62:36541)
0807 08:37:33.917202 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:33.917204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:33.917292 (+ 88us) server_negotiation.cc:407] Connection header received
0807 08:37:33.917352 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:33.917353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:33.917363 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:33.917417 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:33.917912 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.918425 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.919232 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:33.919417 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:33.919476 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:33.921808 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:33.921810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:33.921810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:33.921819 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:33.923375 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.923779 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.923781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.923781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.923847 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:33.924061 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:33.924062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:33.924062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:33.924241 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:33.924360 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:33.924460 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 08:37:33.924555 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 08:37:33.925021 16223 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35750
I20260807 08:37:33.925462 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 16313
I20260807 08:37:33.939996 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 16512
I20260807 08:37:33.950439 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 16707
I20260807 08:37:33.962496 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 16176
2026-08-07T08:37:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2176 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:34 dist-test-slave-h6n0 krb5kdc[16905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:34 dist-test-slave-h6n0 krb5kdc[16905](info): set up 2 sockets
Aug 07 08:37:34 dist-test-slave-h6n0 krb5kdc[16905](info): commencing operation
krb5kdc: starting...
W20260807 08:37:35.225656 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:35.362725 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45169
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:45169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:35.446323 16924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:35.446566 16924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:35.446605 16924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:35.447958 16924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:35.448061 16924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:35.448093 16924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:35.448127 16924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:35.448153 16924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:35.449712 16924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:45169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:35.450016 16924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:35.450263 16924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:35.455309 16930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.455333 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.455309 16932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.455561 16924 server_base.cc:1034] running on GCE node
I20260807 08:37:35.455938 16924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:35.456238 16924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:35.457389 16924 hybrid_clock.cc:648] HybridClock initialized: now 1786091855457351 us; error 54 us; skew 500 ppm
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:35.461032 16924 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:35.461598 16924 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:35.461692 16924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:35.461894 16924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:35.463321 16924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "1e8f6de98ccc7364061adc3af45e331d"
server_key_iv: "6540d4e4523065684fbc4ff8c14daa1d"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.463706 16924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "1e8f6de98ccc7364061adc3af45e331d"
server_key_iv: "6540d4e4523065684fbc4ff8c14daa1d"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.465085 16924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:35.465891 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.466158 16924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:35.466269 16924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "1e8f6de98ccc7364061adc3af45e331d"
server_key_iv: "6540d4e4523065684fbc4ff8c14daa1d"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.466347 16924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:35.476640 16924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:35.478721 16924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:35.478883 16924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:35.483506 16924 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:45169
I20260807 08:37:35.483594 17013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:45169 every 8 connection(s)
I20260807 08:37:35.483852 16924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:35.485005 17014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:35.487901 17014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db: Bootstrap starting.
I20260807 08:37:35.488627 17014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:35.488950 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 16924
I20260807 08:37:35.488991 17014 log.cc:824] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:35.489084 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:35.489310 11068 external_mini_cluster.cc:1428] Setting key 34a547c3a6e6594e2c30f610de741937
I20260807 08:37:35.490061 17014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db: No bootstrap required, opened a new log
I20260807 08:37:35.493124 17014 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45169 } }
I20260807 08:37:35.493319 17014 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:35.493369 17014 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0f6df6aefe4bcca1b5cdb1b83924db, State: Initialized, Role: FOLLOWER
I20260807 08:37:35.493585 17014 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45169 } }
I20260807 08:37:35.493680 17014 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:35.493705 17014 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:35.493759 17014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:35.494616 17014 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45169 } }
I20260807 08:37:35.494743 17014 leader_election.cc:302] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa0f6df6aefe4bcca1b5cdb1b83924db; no voters:
I20260807 08:37:35.494927 17014 leader_election.cc:288] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:35.495020 17019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:35.495324 17019 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [term 1 LEADER]: Becoming Leader. State: Replica: fa0f6df6aefe4bcca1b5cdb1b83924db, State: Running, Role: LEADER
I20260807 08:37:35.495399 17014 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:35.495501 17019 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45169 } }
I20260807 08:37:35.496433 17020 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45169 } } }
I20260807 08:37:35.496551 17020 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:35.496831 17021 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa0f6df6aefe4bcca1b5cdb1b83924db. Latest consensus state: current_term: 1 leader_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0f6df6aefe4bcca1b5cdb1b83924db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45169 } } }
I20260807 08:37:35.496937 17021 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:35.497144 17033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:35.497870 17033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:35.499845 17033 catalog_manager.cc:1347] Generated new cluster ID: 4336841bb2874d9896c53d4a33d3c102
I20260807 08:37:35.499912 17033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:35.502720 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.489895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36518 (local address 127.10.207.62:45169)
0807 08:37:35.490159 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.490169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.490214 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:37:35.490824 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.490835 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.490903 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.491167 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:35.491751 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.493263 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.494209 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.494915 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.495061 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.498252 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:35.498256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:35.498257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:35.498272 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:35.500709 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.501274 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.501277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.501278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.501381 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.501647 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.501648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.501648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.501943 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:35.502085 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.502298 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.502440 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":120,"threads_started":1}
I20260807 08:37:35.510094 17033 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:35.510754 17033 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:35.515903 17033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db: Generated new TSK 0
I20260807 08:37:35.516215 17033 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:35.547621 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45169
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:35.632349 17046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:35.632566 17046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:35.632606 17046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:35.633697 17046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:35.633766 17046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:35.633816 17046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:35.635344 17046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:35.635609 17046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:35.635839 17046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:35.641259 17054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.641288 17057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.641276 17053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.641535 17046 server_base.cc:1034] running on GCE node
I20260807 08:37:35.641968 17046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:35.642266 17046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:35.643461 17046 hybrid_clock.cc:648] HybridClock initialized: now 1786091855643443 us; error 27 us; skew 500 ppm
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:35.646463 17046 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:35.646961 17046 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:35.647042 17046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:35.647192 17046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:35.648394 17046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "0c2ccd776d0e47488b8e81382dd72e16"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "769c359c882e8ffa1d509ad0bdab2b17"
server_key_iv: "83986341f93ed87cbfe4336575a20ec1"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.648847 17046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "0c2ccd776d0e47488b8e81382dd72e16"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "769c359c882e8ffa1d509ad0bdab2b17"
server_key_iv: "83986341f93ed87cbfe4336575a20ec1"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.650311 17046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:35.651301 17067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.651734 17046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:35.651861 17046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "0c2ccd776d0e47488b8e81382dd72e16"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "769c359c882e8ffa1d509ad0bdab2b17"
server_key_iv: "83986341f93ed87cbfe4336575a20ec1"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.651948 17046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:35.677425 17046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:35.679888 17046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:35.680040 17046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:35.680302 17046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:35.680701 17046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:35.680763 17046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.680835 17046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:35.680881 17046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.688551 17046 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:35423
I20260807 08:37:35.688649 17218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:35423 every 8 connection(s)
I20260807 08:37:35.688968 17046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:35.694399 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 17046
I20260807 08:37:35.694568 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:35.694809 11068 external_mini_cluster.cc:1428] Setting key 5cb61fb6a204a5d0377ab0fa9781013d
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:35.700706 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.689833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:41209 (local address 127.10.207.62:45169)
0807 08:37:35.689926 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.689929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.690323 (+ 394us) server_negotiation.cc:407] Connection header received
0807 08:37:35.691033 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.691034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.691046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.691109 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:35.692223 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.692773 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.693935 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.694162 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.694223 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.697446 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:35.697449 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:35.697449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:35.697465 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:35.699115 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.699603 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.699605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.699606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.699671 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.699959 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.699960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.699961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.700096 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:35.700172 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.700496 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.700599 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:37:35.701063 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.689804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45169 (local address 127.10.207.1:41209)
0807 08:37:35.690137 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.690195 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.690914 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.691175 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.691179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.691504 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:35.692100 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.692105 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.692876 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.692876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.693844 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.693846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.694282 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.694284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.694468 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.695257 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:35.695267 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:35.697329 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:35.699233 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.699243 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.699246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.699503 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.699747 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.699749 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.699749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.699876 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.700175 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:35.700184 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:35.700351 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.700474 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.700663 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":141,"threads_started":1}
I20260807 08:37:35.701812 17219 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45169
I20260807 08:37:35.701970 17219 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:35.702189 17219 heartbeater.cc:511] Master 127.10.207.62:45169 requested a full tablet report, sending...
I20260807 08:37:35.702981 16967 ts_manager.cc:194] Registered new tserver with Master: 0c2ccd776d0e47488b8e81382dd72e16 (127.10.207.1:35423)
I20260807 08:37:35.704017 16967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:41209
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:35.725644 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45169
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:35.812820 17225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:35.813094 17225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:35.813140 17225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:35.814291 17225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:35.814352 17225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:35.814393 17225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:35.815979 17225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:35.816217 17225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:35.816485 17225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:35.822184 17239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.822171 17235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.822310 17225 server_base.cc:1034] running on GCE node
W20260807 08:37:35.822171 17236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.822925 17225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:35.823534 17225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:35.824767 17225 hybrid_clock.cc:648] HybridClock initialized: now 1786091855824730 us; error 50 us; skew 500 ppm
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:35.827611 17225 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:35.828006 17225 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:35.828076 17225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:35.828171 17225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:35.829157 17225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "47deb0c220874f9191acbd5b0deffa1c"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "938489669f45d40a8de5d5c336760e88"
server_key_iv: "8860495b75b8da1642215c2337a7d315"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.829475 17225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "47deb0c220874f9191acbd5b0deffa1c"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "938489669f45d40a8de5d5c336760e88"
server_key_iv: "8860495b75b8da1642215c2337a7d315"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.830835 17225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:37:35.831590 17248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.831817 17225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:35.831930 17225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "47deb0c220874f9191acbd5b0deffa1c"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "938489669f45d40a8de5d5c336760e88"
server_key_iv: "8860495b75b8da1642215c2337a7d315"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.832015 17225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:35.846143 17225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:35.848573 17225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:35.848701 17225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:35.848941 17225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:35.849339 17225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:35.849377 17225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.849400 17225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:35.849448 17225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.857355 17225 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:44557
I20260807 08:37:35.857453 17423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:44557 every 8 connection(s)
I20260807 08:37:35.857749 17225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:35.863121 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 17225
I20260807 08:37:35.863344 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:35.863585 11068 external_mini_cluster.cc:1428] Setting key b9aea34cb56ffe20a7cfffe91c5c24a2
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:35.869472 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.858577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:39927 (local address 127.10.207.62:45169)
0807 08:37:35.858667 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:35.858670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:35.858939 (+ 269us) server_negotiation.cc:407] Connection header received
0807 08:37:35.859609 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:35.859611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:35.859625 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:35.859702 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:35.860833 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.861355 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.862502 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.862754 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.862825 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.866029 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:35.866030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:35.866030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:35.866038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:35.867699 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.868269 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.868270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.868271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.868324 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:35.868663 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:35.868664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:35.868665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:35.868842 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:35.868917 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:35.869215 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 08:37:35.869326 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:37:35.869503 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:35.858513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45169 (local address 127.10.207.2:39927)
0807 08:37:35.858768 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:35.858809 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:35.859382 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:35.859820 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:35.859824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:35.860132 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:35.860708 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.860713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.861458 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.861459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.862381 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:35.862384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:35.862849 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:35.862849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:35.862973 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:35.863664 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:35.863674 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:35.865912 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:35.867798 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.867806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.867809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.868183 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.868392 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:35.868393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:35.868394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:35.868529 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:35.868905 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:35.868911 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:35.869057 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 08:37:35.869156 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:37:35.869307 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":137,"threads_started":1}
I20260807 08:37:35.870108 17424 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45169
I20260807 08:37:35.870242 17424 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:35.870430 17424 heartbeater.cc:511] Master 127.10.207.62:45169 requested a full tablet report, sending...
I20260807 08:37:35.871229 16967 ts_manager.cc:194] Registered new tserver with Master: 47deb0c220874f9191acbd5b0deffa1c (127.10.207.2:44557)
I20260807 08:37:35.871821 16967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:39927
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:35.893704 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45169
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:35.978120 17432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:35.978320 17432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:35.978343 17432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:35.979688 17432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:35.979736 17432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:35.979763 17432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:35.981258 17432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:35.981673 17432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:35.982045 17432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:35.988217 17441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.988381 17432 server_base.cc:1034] running on GCE node
W20260807 08:37:35.988191 17444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:35.988296 17440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:35.988932 17432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:35.989259 17432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:35.990461 17432 hybrid_clock.cc:648] HybridClock initialized: now 1786091855990478 us; error 85 us; skew 500 ppm
Aug 07 08:37:35 dist-test-slave-h6n0 krb5kdc[16905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:35.993556 17432 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:35.994000 17432 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:35.994082 17432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:35.994277 17432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:35.995700 17432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "ab094236513f4acab50551c6266c87e2"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "7fe6838b3196cac84822d76acf38c965"
server_key_iv: "b09a5e7ceaa6010b57c0e62d3ac0783b"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.996145 17432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "ab094236513f4acab50551c6266c87e2"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "7fe6838b3196cac84822d76acf38c965"
server_key_iv: "b09a5e7ceaa6010b57c0e62d3ac0783b"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.998013 17432 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:37:35.999011 17454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:35.999300 17432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:35.999424 17432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "ab094236513f4acab50551c6266c87e2"
format_stamp: "Formatted at 2026-08-07 08:37:35 on dist-test-slave-h6n0"
server_key: "7fe6838b3196cac84822d76acf38c965"
server_key_iv: "b09a5e7ceaa6010b57c0e62d3ac0783b"
server_key_version: "encryptionkey@0"
I20260807 08:37:35.999487 17432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.028375 17432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.031072 17432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.031255 17432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.031523 17432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:36.031937 17432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:36.032001 17432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.032049 17432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:36.032078 17432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.039987 17432 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:42681
I20260807 08:37:36.040082 17615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:42681 every 8 connection(s)
I20260807 08:37:36.040419 17432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:37:36 dist-test-slave-h6n0 krb5kdc[16905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091855, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:36.050093 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 17432
I20260807 08:37:36.050220 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:36.050431 11068 external_mini_cluster.cc:1428] Setting key 55cca9a11bbce0e26208fd40e512e34f
I20260807 08:37:36.052011 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.041335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:54345 (local address 127.10.207.62:45169)
0807 08:37:36.041422 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.041426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.041716 (+ 290us) server_negotiation.cc:407] Connection header received
0807 08:37:36.042277 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.042279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.042293 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.042363 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.043493 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.044068 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.045205 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.045430 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.045510 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.048125 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.048127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.048127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:36.048139 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.049966 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:36.050650 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:36.050652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:36.050653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:36.050764 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:36.051140 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:36.051141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:36.051141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:36.051309 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.051379 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.051838 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.051940 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:37:36.052147 17619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.041326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45169 (local address 127.10.207.3:54345)
0807 08:37:36.041564 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.041606 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.042138 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.042454 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.042458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.042822 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.043375 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.043380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.044203 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.044204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.045051 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.045054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.045603 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.045604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.045756 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.046404 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:36.046414 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.047996 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.050088 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:36.050100 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:36.050104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:36.050480 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:36.050864 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:36.050865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:36.050866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:36.051041 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:36.051397 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.051404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.051603 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.051727 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.051893 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
I20260807 08:37:36.052748 17617 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45169
I20260807 08:37:36.052858 17617 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:36.053041 17617 heartbeater.cc:511] Master 127.10.207.62:45169 requested a full tablet report, sending...
I20260807 08:37:36.053570 16967 ts_manager.cc:194] Registered new tserver with Master: ab094236513f4acab50551c6266c87e2 (127.10.207.3:42681)
I20260807 08:37:36.054000 16967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:54345
I20260807 08:37:36.062387 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:36.070832 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.063909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36528 (local address 127.10.207.62:45169)
0807 08:37:36.063994 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.063996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.064092 (+ 96us) server_negotiation.cc:407] Connection header received
0807 08:37:36.064194 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.064195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.064206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.064438 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.065143 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.065674 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.066458 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.066769 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.066850 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.068055 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.068057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.068058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:36.068065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.069552 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:36.069894 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:36.069895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:36.069895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:36.069958 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:36.070165 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:36.070166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:36.070167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:36.070333 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.070447 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.070628 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.070719 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:37:36.072840 16967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:36.074805 16967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:36.082279 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.077568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:44557 (local address 127.0.0.1:36022)
0807 08:37:36.077805 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.077853 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.077924 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.078304 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.078307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.078310 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:36.078398 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.078401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.079950 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.079952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.081110 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.081113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.082005 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.082006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.082077 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.082080 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.082133 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.082179 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":35,"thread_start_us":74,"threads_started":1}
I20260807 08:37:36.082314 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.077808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.10.207.1:35423)
0807 08:37:36.078232 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.078239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.078251 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:37:36.078301 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.078304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.078322 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.078412 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:36.078712 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.079811 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.081133 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.081734 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.081838 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.081902 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.082133 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.082212 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":117,"threads_started":1}
I20260807 08:37:36.082441 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.077678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36022 (local address 127.10.207.2:44557)
0807 08:37:36.078083 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.078088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.078099 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:36.078142 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.078144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.078161 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.078291 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:36.078557 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.079812 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.081212 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.081929 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.082003 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.082054 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.082253 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.082344 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":97,"threads_started":1}
I20260807 08:37:36.082518 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.077859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:42681 (local address 127.0.0.1:56384)
0807 08:37:36.078030 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.078042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.078104 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.078614 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.078615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.078618 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:36.078689 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.078691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.079559 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.079561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.080828 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.080831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.082033 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.082035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.082112 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.082118 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.082185 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.082257 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260807 08:37:36.082581 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.078122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.10.207.3:42681)
0807 08:37:36.078423 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.078427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.078437 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:36.078472 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.078474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.078489 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.078611 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:36.078749 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.079449 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.080942 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.081833 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.081917 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.081988 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.082326 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.082379 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":101,"threads_started":1}
I20260807 08:37:36.082190 17628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.077625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:35423 (local address 127.0.0.1:52176)
0807 08:37:36.077853 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.077863 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.077924 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.078517 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.078518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.078521 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:36.078601 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.078603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.079970 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.079971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.081007 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.081011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.081847 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.081848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.081944 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.081958 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.082030 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.082103 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
I20260807 08:37:36.083505 17321 tablet_service.cc:1467] Processing CreateTablet for tablet 9605b28806b64e0b80ee9e36527d265e (DEFAULT_TABLE table=test-table [id=a20de58e5d9c43bb9f045335ea22e34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:36.083489 17130 tablet_service.cc:1467] Processing CreateTablet for tablet 9605b28806b64e0b80ee9e36527d265e (DEFAULT_TABLE table=test-table [id=a20de58e5d9c43bb9f045335ea22e34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:36.083947 17321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9605b28806b64e0b80ee9e36527d265e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:36.083987 17130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9605b28806b64e0b80ee9e36527d265e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:36.084015 17524 tablet_service.cc:1467] Processing CreateTablet for tablet 9605b28806b64e0b80ee9e36527d265e (DEFAULT_TABLE table=test-table [id=a20de58e5d9c43bb9f045335ea22e34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:36.084355 17524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9605b28806b64e0b80ee9e36527d265e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:36.086889 17638 tablet_bootstrap.cc:492] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Bootstrap starting.
I20260807 08:37:36.087476 17638 tablet_bootstrap.cc:654] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:36.088027 17639 tablet_bootstrap.cc:492] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Bootstrap starting.
I20260807 08:37:36.088116 17638 log.cc:824] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.088303 17640 tablet_bootstrap.cc:492] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16: Bootstrap starting.
I20260807 08:37:36.088968 17640 tablet_bootstrap.cc:654] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:36.089394 17638 tablet_bootstrap.cc:492] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: No bootstrap required, opened a new log
I20260807 08:37:36.089493 17638 ts_tablet_manager.cc:1397] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:36.089600 17640 log.cc:824] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.090397 17640 tablet_bootstrap.cc:492] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16: No bootstrap required, opened a new log
I20260807 08:37:36.090502 17640 ts_tablet_manager.cc:1397] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:37:36.090602 17639 tablet_bootstrap.cc:654] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:36.091202 17639 log.cc:824] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.091773 17638 raft_consensus.cc:348] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.091955 17638 raft_consensus.cc:374] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:36.091995 17638 raft_consensus.cc:729] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47deb0c220874f9191acbd5b0deffa1c, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.092108 17639 tablet_bootstrap.cc:492] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: No bootstrap required, opened a new log
I20260807 08:37:36.092181 17639 ts_tablet_manager.cc:1397] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 08:37:36.092180 17638 consensus_queue.cc:260] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.092545 17638 ts_tablet_manager.cc:1428] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:36.092639 17424 heartbeater.cc:503] Master 127.10.207.62:45169 was elected leader, sending a full tablet report...
I20260807 08:37:36.092757 17640 raft_consensus.cc:348] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.092941 17640 raft_consensus.cc:374] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:36.092981 17640 raft_consensus.cc:729] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c2ccd776d0e47488b8e81382dd72e16, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.093169 17640 consensus_queue.cc:260] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.093719 17640 ts_tablet_manager.cc:1428] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:36.093717 17219 heartbeater.cc:503] Master 127.10.207.62:45169 was elected leader, sending a full tablet report...
I20260807 08:37:36.094285 17639 raft_consensus.cc:348] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.094467 17639 raft_consensus.cc:374] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:36.094507 17639 raft_consensus.cc:729] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab094236513f4acab50551c6266c87e2, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.094671 17639 consensus_queue.cc:260] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.094976 17639 ts_tablet_manager.cc:1428] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 08:37:36.095033 17617 heartbeater.cc:503] Master 127.10.207.62:45169 was elected leader, sending a full tablet report...
W20260807 08:37:36.108297 17426 tablet.cc:2369] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:36.132740 17644 raft_consensus.cc:482] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:36.132858 17644 raft_consensus.cc:504] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.133206 17644 leader_election.cc:288] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c2ccd776d0e47488b8e81382dd72e16 (127.10.207.1:35423), ab094236513f4acab50551c6266c87e2 (127.10.207.3:42681)
I20260807 08:37:36.136726 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.133367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:35423 (local address 127.10.207.2:54261)
0807 08:37:36.133435 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.133448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.133493 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.133806 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.133809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.133812 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:36.133926 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.133929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.134802 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.134803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.135633 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.135636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.136441 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.136442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.136538 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.136541 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.136611 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.136655 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:37:36.136726 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.133509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:54261 (local address 127.10.207.1:35423)
0807 08:37:36.133584 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.133586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.133602 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:37:36.133649 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.133650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.133665 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.133793 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:36.134017 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.134666 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.135781 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.136356 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.136443 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.136453 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.136620 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.136660 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:37:36.137041 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.133622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:44761 (local address 127.10.207.3:42681)
0807 08:37:36.133714 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.133717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.133832 (+ 115us) server_negotiation.cc:407] Connection header received
0807 08:37:36.133912 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.133913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.133926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.133991 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:36.134182 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.134758 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.136151 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.136634 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.136734 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.136748 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.136925 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.136964 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:37:36.137064 17648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.133506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:42681 (local address 127.10.207.2:44761)
0807 08:37:36.133751 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.133760 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.133841 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.134013 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.134014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.134017 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:36.134110 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.134112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.134848 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.134849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.136036 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:36.136039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.136753 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.136755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:36.136840 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.136845 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.136902 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.136948 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":170,"threads_started":1}
I20260807 08:37:36.137110 17156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9605b28806b64e0b80ee9e36527d265e" candidate_uuid: "47deb0c220874f9191acbd5b0deffa1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2ccd776d0e47488b8e81382dd72e16" is_pre_election: true
I20260807 08:37:36.137246 17156 raft_consensus.cc:2454] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47deb0c220874f9191acbd5b0deffa1c in term 0.
I20260807 08:37:36.137507 17251 leader_election.cc:302] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c2ccd776d0e47488b8e81382dd72e16, 47deb0c220874f9191acbd5b0deffa1c; no voters:
I20260807 08:37:36.137672 17644 raft_consensus.cc:2781] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:36.137727 17644 raft_consensus.cc:482] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:36.137753 17644 raft_consensus.cc:3037] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:36.137753 17552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9605b28806b64e0b80ee9e36527d265e" candidate_uuid: "47deb0c220874f9191acbd5b0deffa1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab094236513f4acab50551c6266c87e2" is_pre_election: true
I20260807 08:37:36.137992 17552 raft_consensus.cc:2454] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47deb0c220874f9191acbd5b0deffa1c in term 0.
I20260807 08:37:36.138653 17644 raft_consensus.cc:504] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.138767 17644 leader_election.cc:288] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [CANDIDATE]: Term 1 election: Requested vote from peers 0c2ccd776d0e47488b8e81382dd72e16 (127.10.207.1:35423), ab094236513f4acab50551c6266c87e2 (127.10.207.3:42681)
I20260807 08:37:36.138939 17156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9605b28806b64e0b80ee9e36527d265e" candidate_uuid: "47deb0c220874f9191acbd5b0deffa1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2ccd776d0e47488b8e81382dd72e16"
I20260807 08:37:36.138991 17552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9605b28806b64e0b80ee9e36527d265e" candidate_uuid: "47deb0c220874f9191acbd5b0deffa1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab094236513f4acab50551c6266c87e2"
I20260807 08:37:36.139024 17156 raft_consensus.cc:3037] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:36.139087 17552 raft_consensus.cc:3037] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:36.139813 17156 raft_consensus.cc:2454] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47deb0c220874f9191acbd5b0deffa1c in term 1.
I20260807 08:37:36.140020 17251 leader_election.cc:302] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c2ccd776d0e47488b8e81382dd72e16, 47deb0c220874f9191acbd5b0deffa1c; no voters:
I20260807 08:37:36.140098 17552 raft_consensus.cc:2454] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47deb0c220874f9191acbd5b0deffa1c in term 1.
I20260807 08:37:36.140179 17644 raft_consensus.cc:2781] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:36.140349 17644 raft_consensus.cc:686] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 1 LEADER]: Becoming Leader. State: Replica: 47deb0c220874f9191acbd5b0deffa1c, State: Running, Role: LEADER
I20260807 08:37:36.140431 17644 consensus_queue.cc:237] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } }
I20260807 08:37:36.141325 16966 catalog_manager.cc:5495] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 47deb0c220874f9191acbd5b0deffa1c (127.10.207.2). New cstate: current_term: 1 leader_uuid: "47deb0c220874f9191acbd5b0deffa1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47deb0c220874f9191acbd5b0deffa1c" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:36.156078 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.153599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36036 (local address 127.10.207.2:44557)
0807 08:37:36.153654 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.153655 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.153666 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:36.153724 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.153724 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.153741 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.153806 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:36.153967 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.154388 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.155309 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:36.155506 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:36.155567 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:36.155720 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:36.155804 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:36.155950 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.155998 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.156017 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:37:36.158156 17156 raft_consensus.cc:1261] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 1 FOLLOWER]: Refusing update from remote peer 47deb0c220874f9191acbd5b0deffa1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:36.158157 17552 raft_consensus.cc:1261] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 1 FOLLOWER]: Refusing update from remote peer 47deb0c220874f9191acbd5b0deffa1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:36.158511 17644 consensus_queue.cc:1035] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c2ccd776d0e47488b8e81382dd72e16" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:36.158689 17649 consensus_queue.cc:1035] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab094236513f4acab50551c6266c87e2" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:36.162429 17654 mvcc.cc:205] Tried to move back new op lower bound from 7315832242820808704 to 7315832242751705088. Current Snapshot: MvccSnapshot[applied={T|T < 7315832242820808704 or (T in {7315832242820808704})}]
I20260807 08:37:36.164850 16966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:36.164999 16966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:36.166115 16966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa0f6df6aefe4bcca1b5cdb1b83924db: Sending DeleteTablet for 3 replicas of tablet 9605b28806b64e0b80ee9e36527d265e
I20260807 08:37:36.166604 17130 tablet_service.cc:1514] Processing DeleteTablet for tablet 9605b28806b64e0b80ee9e36527d265e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:36 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:52176
I20260807 08:37:36.166635 17524 tablet_service.cc:1514] Processing DeleteTablet for tablet 9605b28806b64e0b80ee9e36527d265e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:36 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:56384
I20260807 08:37:36.166674 17321 tablet_service.cc:1514] Processing DeleteTablet for tablet 9605b28806b64e0b80ee9e36527d265e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:36 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:36022
I20260807 08:37:36.167032 17668 tablet_replica.cc:330] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16: stopping tablet replica
I20260807 08:37:36.167164 17668 raft_consensus.cc:2229] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:36.167297 17668 raft_consensus.cc:2258] T 9605b28806b64e0b80ee9e36527d265e P 0c2ccd776d0e47488b8e81382dd72e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:36.167305 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 17046
I20260807 08:37:36.170500 17669 tablet_replica.cc:330] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: stopping tablet replica
I20260807 08:37:36.170640 17669 raft_consensus.cc:2229] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:36.170761 17669 raft_consensus.cc:2258] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:36.171088 17670 tablet_replica.cc:330] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: stopping tablet replica
I20260807 08:37:36.171211 17670 raft_consensus.cc:2229] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:36.171370 17670 raft_consensus.cc:2258] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:36.173022 17669 ts_tablet_manager.cc:1905] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:36.173251 17670 ts_tablet_manager.cc:1905] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:36.175128 17669 ts_tablet_manager.cc:1918] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:36.175184 17670 ts_tablet_manager.cc:1918] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:36.175236 17669 log.cc:1196] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/9605b28806b64e0b80ee9e36527d265e
I20260807 08:37:36.175292 17670 log.cc:1196] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/9605b28806b64e0b80ee9e36527d265e
I20260807 08:37:36.175624 17669 ts_tablet_manager.cc:1939] T 9605b28806b64e0b80ee9e36527d265e P ab094236513f4acab50551c6266c87e2: Deleting consensus metadata
I20260807 08:37:36.175648 17670 ts_tablet_manager.cc:1939] T 9605b28806b64e0b80ee9e36527d265e P 47deb0c220874f9191acbd5b0deffa1c: Deleting consensus metadata
I20260807 08:37:36.176244 16944 catalog_manager.cc:4841] TS 47deb0c220874f9191acbd5b0deffa1c (127.10.207.2:44557): tablet 9605b28806b64e0b80ee9e36527d265e (table test-table [id=a20de58e5d9c43bb9f045335ea22e34f]) successfully deleted
I20260807 08:37:36.176244 16945 catalog_manager.cc:4841] TS ab094236513f4acab50551c6266c87e2 (127.10.207.3:42681): tablet 9605b28806b64e0b80ee9e36527d265e (table test-table [id=a20de58e5d9c43bb9f045335ea22e34f]) successfully deleted
W20260807 08:37:36.183138 16945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:35423 (error 108)
I20260807 08:37:36.183332 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 17225
I20260807 08:37:36.194919 17628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.188740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:35423 (local address 127.0.0.1:52192)
0807 08:37:36.194785 (+ 6045us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.194873 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6008}
W20260807 08:37:36.195092 16945 catalog_manager.cc:4584] TS 0c2ccd776d0e47488b8e81382dd72e16 (127.10.207.1:35423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9605b28806b64e0b80ee9e36527d265e: Network error: Client connection negotiation failed: client connection to 127.10.207.1:35423: connect: Connection refused (error 111)
I20260807 08:37:36.205384 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 17432
I20260807 08:37:36.220649 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 16924
2026-08-07T08:37:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2257 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T08:37:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:36Z Disabled control of system clock
I20260807 08:37:36.259452 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35201
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:35201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:37:36.349282 17679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:36.349495 17679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:36.349520 17679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:36.350608 17679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:36.350648 17679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:36.350661 17679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:36.350674 17679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:36.350687 17679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:36.351967 17679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:35201
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:36.352169 17679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:36.352403 17679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:36.357671 17685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.357646 17686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.357776 17688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:36.358114 17679 server_base.cc:1034] running on GCE node
I20260807 08:37:36.358410 17679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:36.358795 17679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:36.359987 17679 hybrid_clock.cc:648] HybridClock initialized: now 1786091856359971 us; error 48 us; skew 500 ppm
I20260807 08:37:36.361637 17679 webserver.cc:460] Webserver started at http://127.10.207.62:41265/ using document root <none> and password file <none>
I20260807 08:37:36.361935 17679 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:36.362008 17679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:36.362174 17679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:36.363471 17679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "2656aa5da46e4afeae699354ca4a2924"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "46e9c13c72761eea35b408111c1db509"
server_key_iv: "e9c0ec657319cca8213989190aa50e9d"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.363873 17679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "2656aa5da46e4afeae699354ca4a2924"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "46e9c13c72761eea35b408111c1db509"
server_key_iv: "e9c0ec657319cca8213989190aa50e9d"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.365378 17679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:36.366302 17696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.366619 17679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:36.366742 17679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "2656aa5da46e4afeae699354ca4a2924"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "46e9c13c72761eea35b408111c1db509"
server_key_iv: "e9c0ec657319cca8213989190aa50e9d"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.366837 17679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.370602 17679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.370915 17679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.371071 17679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.375895 17679 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:35201
I20260807 08:37:36.375962 17765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:35201 every 8 connection(s)
I20260807 08:37:36.376348 17679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:36.377164 17766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:36.380929 17766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924: Bootstrap starting.
I20260807 08:37:36.381778 17766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:36.382151 17766 log.cc:824] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.383195 17766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924: No bootstrap required, opened a new log
I20260807 08:37:36.385566 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 17679
I20260807 08:37:36.385723 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:36.385632 17766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2656aa5da46e4afeae699354ca4a2924" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35201 } }
I20260807 08:37:36.385900 17766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:36.385977 11068 external_mini_cluster.cc:1428] Setting key 6cc3eb16585c34c01f9e223b36379f23
I20260807 08:37:36.385970 17766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2656aa5da46e4afeae699354ca4a2924, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.386147 17766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2656aa5da46e4afeae699354ca4a2924" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35201 } }
I20260807 08:37:36.386303 17766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:36.386346 17766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:36.386404 17766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:36.387635 17766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2656aa5da46e4afeae699354ca4a2924" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35201 } }
I20260807 08:37:36.387884 17766 leader_election.cc:302] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2656aa5da46e4afeae699354ca4a2924; no voters:
I20260807 08:37:36.388118 17766 leader_election.cc:288] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:36.388245 17774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:36.388456 17774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [term 1 LEADER]: Becoming Leader. State: Replica: 2656aa5da46e4afeae699354ca4a2924, State: Running, Role: LEADER
I20260807 08:37:36.388463 17766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:36.388619 17774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2656aa5da46e4afeae699354ca4a2924" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35201 } }
I20260807 08:37:36.388947 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.386631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.10.207.62:35201)
0807 08:37:36.386978 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.386989 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.387031 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:37:36.387168 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.387175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.387220 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.387414 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.388268 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.388272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.388272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.388277 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.388326 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.388343 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.388404 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.388644 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.388710 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":130,"threads_started":1}
I20260807 08:37:36.389997 17775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2656aa5da46e4afeae699354ca4a2924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2656aa5da46e4afeae699354ca4a2924" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35201 } } }
I20260807 08:37:36.390144 17775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:36.389981 17776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2656aa5da46e4afeae699354ca4a2924. Latest consensus state: current_term: 1 leader_uuid: "2656aa5da46e4afeae699354ca4a2924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2656aa5da46e4afeae699354ca4a2924" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35201 } } }
I20260807 08:37:36.390290 17776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:36.390396 17789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:36.391188 17789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:36.393030 17789 catalog_manager.cc:1347] Generated new cluster ID: 92ae1717c6c5447fa8f6b72461bf97ef
I20260807 08:37:36.393087 17789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:36.402966 17789 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:36.403175 17789 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:36.407055 17789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924: Generated new TSK 0
I20260807 08:37:36.407259 17789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:37:36.410430 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35201
--builtin_ntp_servers=127.10.207.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:37:36.505599 17798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:36.505954 17798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:36.506004 17798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:36.507750 17798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:36.507843 17798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:36.507899 17798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:36.509676 17798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:36.510032 17798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:36.510456 17798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:36.516952 17806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.516952 17807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.517100 17809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:36.517469 17798 server_base.cc:1034] running on GCE node
I20260807 08:37:36.517771 17798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:36.518098 17798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:36.519296 17798 hybrid_clock.cc:648] HybridClock initialized: now 1786091856519263 us; error 49 us; skew 500 ppm
I20260807 08:37:36.520900 17798 webserver.cc:460] Webserver started at http://127.10.207.1:43273/ using document root <none> and password file <none>
I20260807 08:37:36.521183 17798 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:36.521267 17798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:36.521435 17798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:36.522780 17798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "ba9f9aa320c744ada3b1e27a68936192"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "c6f25d7ca986952e143b3d24863838ce"
server_key_iv: "63fe37a450754831353ead02b4f23a87"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.523191 17798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "ba9f9aa320c744ada3b1e27a68936192"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "c6f25d7ca986952e143b3d24863838ce"
server_key_iv: "63fe37a450754831353ead02b4f23a87"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.524816 17798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:36.525758 17818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.526015 17798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:36.526142 17798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "ba9f9aa320c744ada3b1e27a68936192"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "c6f25d7ca986952e143b3d24863838ce"
server_key_iv: "63fe37a450754831353ead02b4f23a87"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.526226 17798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.530161 17798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.530499 17798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.530682 17798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.530944 17798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:36.531322 17798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:36.531380 17798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.531415 17798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:36.531441 17798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.538856 17798 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:36863
I20260807 08:37:36.538914 17965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:36863 every 8 connection(s)
I20260807 08:37:36.539299 17798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:36.543000 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.540129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:50313 (local address 127.10.207.62:35201)
0807 08:37:36.540202 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.540205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.540456 (+ 251us) server_negotiation.cc:407] Connection header received
0807 08:37:36.541131 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.541133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.541144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.541200 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.542686 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.542687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.542688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.542693 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.542711 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.542717 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.542790 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.542896 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.542917 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:37:36.543249 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.540077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35201 (local address 127.10.207.1:50313)
0807 08:37:36.540302 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.540342 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.540997 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.541287 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.541291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.541300 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.542455 (+ 1155us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.542456 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.542488 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.542522 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.542786 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.542794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.542798 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.542895 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.542966 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260807 08:37:36.543965 17966 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35201
I20260807 08:37:36.544075 17966 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:36.544324 17966 heartbeater.cc:511] Master 127.10.207.62:35201 requested a full tablet report, sending...
I20260807 08:37:36.545022 17723 ts_manager.cc:194] Registered new tserver with Master: ba9f9aa320c744ada3b1e27a68936192 (127.10.207.1:36863)
I20260807 08:37:36.547544 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 17798
I20260807 08:37:36.547647 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:36.547818 11068 external_mini_cluster.cc:1428] Setting key ecd8775683acbf043e11170eac1212e4
I20260807 08:37:36.548965 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35201
--builtin_ntp_servers=127.10.207.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:37:36.647837 17969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:36.648185 17969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:36.648231 17969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:36.649600 17969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:36.649672 17969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:36.649720 17969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:36.651260 17969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:36.651615 17969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:36.652009 17969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:36.658236 17975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.658215 17977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.658215 17979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:36.658258 17969 server_base.cc:1034] running on GCE node
I20260807 08:37:36.658777 17969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:36.659025 17969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:36.660226 17969 hybrid_clock.cc:648] HybridClock initialized: now 1786091856660201 us; error 37 us; skew 500 ppm
I20260807 08:37:36.662377 17969 webserver.cc:460] Webserver started at http://127.10.207.2:43421/ using document root <none> and password file <none>
I20260807 08:37:36.662794 17969 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:36.662858 17969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:36.663046 17969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:36.664373 17969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "cf78c99c419c187e97f8514a9d9ecdce"
server_key_iv: "592effde784b27e6af8c44d46bac3ca3"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.664819 17969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "cf78c99c419c187e97f8514a9d9ecdce"
server_key_iv: "592effde784b27e6af8c44d46bac3ca3"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.666819 17969 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260807 08:37:36.667974 17985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.668326 17969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:36.668469 17969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "cf78c99c419c187e97f8514a9d9ecdce"
server_key_iv: "592effde784b27e6af8c44d46bac3ca3"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.668572 17969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.672403 17969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.672698 17969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.672863 17969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.673100 17969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:36.673462 17969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:36.673523 17969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.673557 17969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:36.673584 17969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.680469 17969 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:33923
I20260807 08:37:36.680645 18145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:33923 every 8 connection(s)
I20260807 08:37:36.680908 17969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:36.685189 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.682012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:46047 (local address 127.10.207.62:35201)
0807 08:37:36.682149 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.682152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.682405 (+ 253us) server_negotiation.cc:407] Connection header received
0807 08:37:36.683439 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.683441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.683448 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.683501 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.684886 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.684888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.684888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.684893 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.684927 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.684932 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.684993 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.685100 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.685117 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:37:36.685421 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.681965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35201 (local address 127.10.207.2:46047)
0807 08:37:36.682265 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.682302 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.683093 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.683585 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.683589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.683597 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.684712 (+ 1115us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.684713 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.684742 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.684776 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.685004 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.685011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.685014 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.685111 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.685173 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":135,"threads_started":1}
I20260807 08:37:36.685782 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 17969
I20260807 08:37:36.685896 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:36.686122 11068 external_mini_cluster.cc:1428] Setting key e552e3b66bb63254bdd27b60b7b4e7e4
I20260807 08:37:36.686189 18146 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35201
I20260807 08:37:36.686353 18146 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:36.686578 18146 heartbeater.cc:511] Master 127.10.207.62:35201 requested a full tablet report, sending...
I20260807 08:37:36.687222 17723 ts_manager.cc:194] Registered new tserver with Master: 69fe85fc81c14ec5baa95ab01cbc4b3f (127.10.207.2:33923)
I20260807 08:37:36.687558 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35201
--builtin_ntp_servers=127.10.207.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:37:36.781999 18149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:36.782275 18149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:36.782301 18149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:36.783717 18149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:36.783779 18149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:36.783810 18149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:36.785312 18149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:36.785658 18149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:36.786057 18149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:36.792470 18157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.792475 18159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:36.792515 18156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:36.792876 18149 server_base.cc:1034] running on GCE node
I20260807 08:37:36.793164 18149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:36.793467 18149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:36.794636 18149 hybrid_clock.cc:648] HybridClock initialized: now 1786091856794594 us; error 43 us; skew 500 ppm
I20260807 08:37:36.796353 18149 webserver.cc:460] Webserver started at http://127.10.207.3:44831/ using document root <none> and password file <none>
I20260807 08:37:36.796681 18149 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:36.796759 18149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:36.796931 18149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:36.798375 18149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "4f092982065e45898d27a72c010350ca"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "cc83b6425f2a601e9bc3a617f4511b78"
server_key_iv: "5bf413d45c3fb0a88b6aa8b1a1a687df"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.798885 18149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "4f092982065e45898d27a72c010350ca"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "cc83b6425f2a601e9bc3a617f4511b78"
server_key_iv: "5bf413d45c3fb0a88b6aa8b1a1a687df"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.800784 18149 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:37:36.801949 18169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.802222 18149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:36.802359 18149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "4f092982065e45898d27a72c010350ca"
format_stamp: "Formatted at 2026-08-07 08:37:36 on dist-test-slave-h6n0"
server_key: "cc83b6425f2a601e9bc3a617f4511b78"
server_key_iv: "5bf413d45c3fb0a88b6aa8b1a1a687df"
server_key_version: "encryptionkey@0"
I20260807 08:37:36.802455 18149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:36.806744 18149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:36.807103 18149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:36.807277 18149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:36.807536 18149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:36.807937 18149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:36.807994 18149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.808056 18149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:36.808084 18149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:36.815773 18149 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:32863
I20260807 08:37:36.815819 18328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:32863 every 8 connection(s)
I20260807 08:37:36.816244 18149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:36.819870 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.817219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:53029 (local address 127.10.207.62:35201)
0807 08:37:36.817288 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.817291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.817587 (+ 296us) server_negotiation.cc:407] Connection header received
0807 08:37:36.818190 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.818191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.818199 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.818252 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.819575 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.819577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.819577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.819582 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.819597 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.819601 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.819666 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.819773 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.819791 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:37:36.820108 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.817170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35201 (local address 127.10.207.3:53029)
0807 08:37:36.817434 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.817479 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.818066 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.818333 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.818337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.818357 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.819404 (+ 1047us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.819404 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.819436 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.819468 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.819670 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.819675 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.819679 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.819768 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.819836 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":106,"threads_started":1}
I20260807 08:37:36.820747 18331 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35201
I20260807 08:37:36.820859 18331 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:36.821040 18331 heartbeater.cc:511] Master 127.10.207.62:35201 requested a full tablet report, sending...
I20260807 08:37:36.821496 17723 ts_manager.cc:194] Registered new tserver with Master: 4f092982065e45898d27a72c010350ca (127.10.207.3:32863)
I20260807 08:37:36.824285 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 18149
I20260807 08:37:36.824398 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:36.824594 11068 external_mini_cluster.cc:1428] Setting key e6a99c6875004a34b1e98c3dde7b3152
I20260807 08:37:36.825564 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:36.828300 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.827057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.10.207.62:35201)
0807 08:37:36.827125 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.827127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.827304 (+ 177us) server_negotiation.cc:407] Connection header received
0807 08:37:36.827385 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.827386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.827391 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.827446 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.828092 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.828094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.828094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.828098 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.828113 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.828117 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.828166 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.828221 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.828236 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:37:36.830149 17723 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:36.832239 17723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:36.837009 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.835436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41926 (local address 127.10.207.3:32863)
0807 08:37:36.835642 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.835647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.835658 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:36.835682 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.835684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.835693 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.835787 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.836716 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.836722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.836722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.836726 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.836761 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.836779 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.836837 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.836918 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.836933 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":99,"threads_started":1}
I20260807 08:37:36.836999 18345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.835335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:32863 (local address 127.0.0.1:41926)
0807 08:37:36.835556 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.835571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.835618 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.835791 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.835798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.835804 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.836580 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.836581 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.836600 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.836625 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.836831 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.836838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.836841 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.836914 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.836929 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":130,"threads_started":1}
I20260807 08:37:36.837843 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.835006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36863 (local address 127.0.0.1:34254)
0807 08:37:36.835209 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.835255 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.835321 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.835951 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.835952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.835956 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.837031 (+ 1075us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.837032 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.837037 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.837052 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.837691 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.837692 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.837692 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.837743 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.837760 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
I20260807 08:37:36.837855 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.835163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.10.207.1:36863)
0807 08:37:36.835737 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.835745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.835760 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:37:36.835796 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.835799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.835811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.837215 (+ 1404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.837555 (+ 340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.837560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.837560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.837564 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.837606 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.837624 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.837699 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.837751 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.837766 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":122,"threads_started":1}
I20260807 08:37:36.837936 18231 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 (DEFAULT_TABLE table=test-table [id=dd37350f781545b38bd1bb65ae73b74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:36.838223 18350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.835948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.10.207.2:33923)
0807 08:37:36.836198 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.836243 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.836322 (+ 79us) server_negotiation.cc:407] Connection header received
0807 08:37:36.836392 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.836395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.836406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.836495 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.837668 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.837675 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.837675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.837679 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.837711 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.837731 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.837804 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.838049 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.838066 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":115,"threads_started":1}
I20260807 08:37:36.838344 18231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:36.838981 17882 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 (DEFAULT_TABLE table=test-table [id=dd37350f781545b38bd1bb65ae73b74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:36.839398 17882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:36.841174 18352 tablet_bootstrap.cc:492] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Bootstrap starting.
I20260807 08:37:36.842061 18352 tablet_bootstrap.cc:654] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:36.842458 18352 log.cc:824] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.837986 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.835687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:33923 (local address 127.0.0.1:49110)
0807 08:37:36.836245 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.836258 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.836321 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.836526 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.836527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.836531 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.837023 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.837024 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.837035 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.837574 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.837839 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.837841 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.837841 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.837888 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.837907 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":88,"threads_started":1}
I20260807 08:37:36.843078 18354 tablet_bootstrap.cc:492] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192: Bootstrap starting.
I20260807 08:37:36.843281 18045 tablet_service.cc:1467] Processing CreateTablet for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 (DEFAULT_TABLE table=test-table [id=dd37350f781545b38bd1bb65ae73b74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:36.843672 18045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:36.843961 18354 tablet_bootstrap.cc:654] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:36.844300 18354 log.cc:824] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.845148 18354 tablet_bootstrap.cc:492] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192: No bootstrap required, opened a new log
I20260807 08:37:36.845202 18352 tablet_bootstrap.cc:492] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: No bootstrap required, opened a new log
I20260807 08:37:36.845243 18354 ts_tablet_manager.cc:1397] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:37:36.845288 18352 ts_tablet_manager.cc:1397] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 08:37:36.847278 18356 tablet_bootstrap.cc:492] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Bootstrap starting.
I20260807 08:37:36.847934 18352 raft_consensus.cc:348] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.848090 18352 raft_consensus.cc:374] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:36.847920 18354 raft_consensus.cc:348] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.848114 18352 raft_consensus.cc:729] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f092982065e45898d27a72c010350ca, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.848140 18354 raft_consensus.cc:374] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:36.848181 18354 raft_consensus.cc:729] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba9f9aa320c744ada3b1e27a68936192, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.848194 18352 consensus_queue.cc:260] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.848308 18354 consensus_queue.cc:260] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.848448 18352 ts_tablet_manager.cc:1428] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260807 08:37:36.848465 18331 heartbeater.cc:503] Master 127.10.207.62:35201 was elected leader, sending a full tablet report...
I20260807 08:37:36.848665 18354 ts_tablet_manager.cc:1428] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 08:37:36.848745 18356 tablet_bootstrap.cc:654] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:36.848824 17966 heartbeater.cc:503] Master 127.10.207.62:35201 was elected leader, sending a full tablet report...
I20260807 08:37:36.849103 18356 log.cc:824] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:36.850366 18356 tablet_bootstrap.cc:492] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: No bootstrap required, opened a new log
I20260807 08:37:36.850448 18356 ts_tablet_manager.cc:1397] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:36.852599 18356 raft_consensus.cc:348] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.852757 18356 raft_consensus.cc:374] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:36.852782 18356 raft_consensus.cc:729] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69fe85fc81c14ec5baa95ab01cbc4b3f, State: Initialized, Role: FOLLOWER
I20260807 08:37:36.852936 18356 consensus_queue.cc:260] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.853248 18356 ts_tablet_manager.cc:1428] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:36.853317 18146 heartbeater.cc:503] Master 127.10.207.62:35201 was elected leader, sending a full tablet report...
I20260807 08:37:36.926851 18358 raft_consensus.cc:482] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:36.927098 18358 raft_consensus.cc:504] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.927524 18358 leader_election.cc:288] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f092982065e45898d27a72c010350ca (127.10.207.3:32863), 69fe85fc81c14ec5baa95ab01cbc4b3f (127.10.207.2:33923)
I20260807 08:37:36.929246 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.927740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:32863 (local address 127.10.207.1:56791)
0807 08:37:36.927820 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.927838 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.927901 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.928102 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.928103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.928107 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.928719 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.928719 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.928731 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.928751 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.929035 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.929037 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.929038 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.929100 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.929152 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 08:37:36.929355 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.927869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:56791 (local address 127.10.207.3:32863)
0807 08:37:36.927941 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.927943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.927953 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:36.927984 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.927985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.927991 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.928095 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.928883 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.928886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.928886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.928893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.928920 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.928927 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.929012 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.929253 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.929271 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:37:36.929378 18350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.928020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:46719 (local address 127.10.207.2:33923)
0807 08:37:36.928095 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.928098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.928155 (+ 57us) server_negotiation.cc:407] Connection header received
0807 08:37:36.928240 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.928241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.928247 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.928304 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.929072 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.929074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.929074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.929081 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.929130 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.929137 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.929201 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.929289 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.929307 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:37:36.929392 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.927871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:33923 (local address 127.10.207.1:46719)
0807 08:37:36.928095 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.928108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:36.928173 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:36.928316 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:36.928317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:36.928319 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:36.928928 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:37:36.928929 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:36.928940 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:37:36.928958 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:36.929206 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:36.929207 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:36.929208 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:37:36.929278 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:37:36.929293 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":106,"threads_started":1}
I20260807 08:37:36.929667 18260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7d3c7ff3ce4d9faf0990b0bd1bea21" candidate_uuid: "ba9f9aa320c744ada3b1e27a68936192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f092982065e45898d27a72c010350ca" is_pre_election: true
I20260807 08:37:36.929888 18260 raft_consensus.cc:2454] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba9f9aa320c744ada3b1e27a68936192 in term 0.
I20260807 08:37:36.930023 18079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7d3c7ff3ce4d9faf0990b0bd1bea21" candidate_uuid: "ba9f9aa320c744ada3b1e27a68936192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" is_pre_election: true
I20260807 08:37:36.930236 18079 raft_consensus.cc:2454] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba9f9aa320c744ada3b1e27a68936192 in term 0.
I20260807 08:37:36.930236 17822 leader_election.cc:302] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f092982065e45898d27a72c010350ca, ba9f9aa320c744ada3b1e27a68936192; no voters:
I20260807 08:37:36.930373 18358 raft_consensus.cc:2781] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:36.930450 18358 raft_consensus.cc:482] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:36.930480 18358 raft_consensus.cc:3037] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 0 FOLLOWER]: Advancing to term 1
W20260807 08:37:36.931496 18147 tablet.cc:2369] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:36.931504 18358 raft_consensus.cc:504] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.931668 18358 leader_election.cc:288] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [CANDIDATE]: Term 1 election: Requested vote from peers 4f092982065e45898d27a72c010350ca (127.10.207.3:32863), 69fe85fc81c14ec5baa95ab01cbc4b3f (127.10.207.2:33923)
I20260807 08:37:36.931825 18260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7d3c7ff3ce4d9faf0990b0bd1bea21" candidate_uuid: "ba9f9aa320c744ada3b1e27a68936192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f092982065e45898d27a72c010350ca"
I20260807 08:37:36.931852 18079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e7d3c7ff3ce4d9faf0990b0bd1bea21" candidate_uuid: "ba9f9aa320c744ada3b1e27a68936192" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f"
I20260807 08:37:36.931910 18260 raft_consensus.cc:3037] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:36.931928 18079 raft_consensus.cc:3037] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:36.932711 18079 raft_consensus.cc:2454] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba9f9aa320c744ada3b1e27a68936192 in term 1.
I20260807 08:37:36.932729 18260 raft_consensus.cc:2454] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba9f9aa320c744ada3b1e27a68936192 in term 1.
I20260807 08:37:36.932938 17822 leader_election.cc:302] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f092982065e45898d27a72c010350ca, ba9f9aa320c744ada3b1e27a68936192; no voters:
I20260807 08:37:36.933073 18358 raft_consensus.cc:2781] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:36.933257 18358 raft_consensus.cc:686] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [term 1 LEADER]: Becoming Leader. State: Replica: ba9f9aa320c744ada3b1e27a68936192, State: Running, Role: LEADER
I20260807 08:37:36.933378 18358 consensus_queue.cc:237] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } }
I20260807 08:37:36.934299 17723 catalog_manager.cc:5495] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba9f9aa320c744ada3b1e27a68936192 (127.10.207.1). New cstate: current_term: 1 leader_uuid: "ba9f9aa320c744ada3b1e27a68936192" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba9f9aa320c744ada3b1e27a68936192" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:36.953711 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.952537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.10.207.1:36863)
0807 08:37:36.952617 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:36.952620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:36.952629 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:36.952676 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:36.952677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:36.952696 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:36.952777 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:36.953481 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:36.953482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:36.953483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:36.953487 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:36.953505 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:36.953510 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:36.953560 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:36.953606 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 08:37:36.953622 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:37:36.955912 18079 raft_consensus.cc:1261] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 1 FOLLOWER]: Refusing update from remote peer ba9f9aa320c744ada3b1e27a68936192: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:36.956111 18260 raft_consensus.cc:1261] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 1 FOLLOWER]: Refusing update from remote peer ba9f9aa320c744ada3b1e27a68936192: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:36.956372 18362 consensus_queue.cc:1035] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69fe85fc81c14ec5baa95ab01cbc4b3f" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:36.956527 18358 consensus_queue.cc:1035] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P ba9f9aa320c744ada3b1e27a68936192 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f092982065e45898d27a72c010350ca" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 32863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:36.959374 18366 mvcc.cc:205] Tried to move back new op lower bound from 7315832246088560640 to 7315832245999890432. Current Snapshot: MvccSnapshot[applied={T|T < 7315832246088560640}]
I20260807 08:37:36.959448 18367 mvcc.cc:205] Tried to move back new op lower bound from 7315832246088560640 to 7315832245999890432. Current Snapshot: MvccSnapshot[applied={T|T < 7315832246088560640}]
I20260807 08:37:36.959846 18368 mvcc.cc:205] Tried to move back new op lower bound from 7315832246088560640 to 7315832245999890432. Current Snapshot: MvccSnapshot[applied={T|T < 7315832246088560640}]
I20260807 08:37:36.962370 17721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:36.962512 17721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:36.963516 17721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2656aa5da46e4afeae699354ca4a2924: Sending DeleteTablet for 3 replicas of tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21
I20260807 08:37:36.963883 17882 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:36 UTC) from {username='slave'} at 127.0.0.1:34254
I20260807 08:37:36.964020 18045 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:36 UTC) from {username='slave'} at 127.0.0.1:49110
I20260807 08:37:36.964102 18231 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:36 UTC) from {username='slave'} at 127.0.0.1:41926
I20260807 08:37:36.964339 18385 tablet_replica.cc:330] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: stopping tablet replica
I20260807 08:37:36.964380 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 17798
I20260807 08:37:36.964459 18385 raft_consensus.cc:2229] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:36.964581 18385 raft_consensus.cc:2258] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:36.965162 18385 ts_tablet_manager.cc:1905] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:36.967768 18385 ts_tablet_manager.cc:1918] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:36.967844 18385 log.cc:1196] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/3e7d3c7ff3ce4d9faf0990b0bd1bea21
I20260807 08:37:36.968130 18385 ts_tablet_manager.cc:1939] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 69fe85fc81c14ec5baa95ab01cbc4b3f: Deleting consensus metadata
I20260807 08:37:36.968576 17699 catalog_manager.cc:4841] TS 69fe85fc81c14ec5baa95ab01cbc4b3f (127.10.207.2:33923): tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 (table test-table [id=dd37350f781545b38bd1bb65ae73b74b]) successfully deleted
I20260807 08:37:36.978751 18387 tablet_replica.cc:330] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: stopping tablet replica
I20260807 08:37:36.978973 18387 raft_consensus.cc:2229] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:36.979175 18387 raft_consensus.cc:2258] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:36.979852 18387 ts_tablet_manager.cc:1905] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:37:36.981432 17699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:36863 (error 108)
I20260807 08:37:36.981585 18387 ts_tablet_manager.cc:1918] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:36.981673 18387 log.cc:1196] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/3e7d3c7ff3ce4d9faf0990b0bd1bea21
I20260807 08:37:36.981781 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 17969
I20260807 08:37:36.981984 18387 ts_tablet_manager.cc:1939] T 3e7d3c7ff3ce4d9faf0990b0bd1bea21 P 4f092982065e45898d27a72c010350ca: Deleting consensus metadata
I20260807 08:37:36.986254 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:36.982009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36863 (local address 127.0.0.1:34280)
0807 08:37:36.986135 (+ 4126us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:36.986209 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4080}
W20260807 08:37:36.986421 17699 catalog_manager.cc:4584] TS ba9f9aa320c744ada3b1e27a68936192 (127.10.207.1:36863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36863: connect: Connection refused (error 111)
I20260807 08:37:36.990991 17699 catalog_manager.cc:4841] TS 4f092982065e45898d27a72c010350ca (127.10.207.3:32863): tablet 3e7d3c7ff3ce4d9faf0990b0bd1bea21 (table test-table [id=dd37350f781545b38bd1bb65ae73b74b]) successfully deleted
I20260807 08:37:37.004529 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 18149
I20260807 08:37:37.013896 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 17679
2026-08-07T08:37:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (795 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 08:37:37.044142 11068 mini_oidc.cc:152] Starting JWKS server
I20260807 08:37:37.045367 11068 webserver.cc:460] Webserver started at https://127.0.0.1:33573/ using document root <none> and password file <none>
I20260807 08:37:37.045500 11068 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:37:37.045689 11068 webserver.cc:460] Webserver started at http://127.0.0.1:44261/ using document root <none> and password file <none>
2026-08-07T08:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:37Z Disabled control of system clock
I20260807 08:37:37.055786 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:33849
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:33239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:33849
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33573/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/testchainca.cert with env {}
W20260807 08:37:37.143704 18397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:37.143978 18397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:37.144013 18397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:37.145727 18397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:37.145814 18397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:37.145829 18397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:37.145843 18397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:37.145859 18397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:37:37.145874 18397 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:37:37.145885 18397 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33573/jwks.json
I20260807 08:37:37.148073 18397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:33849
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:33849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33573/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:37.148288 18397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:37.148461 18397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:37.154412 18404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.154469 18406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.154484 18403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:37.155102 18397 server_base.cc:1034] running on GCE node
I20260807 08:37:37.155485 18397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:37.155798 18397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:37.156960 18397 hybrid_clock.cc:648] HybridClock initialized: now 1786091857156925 us; error 51 us; skew 500 ppm
I20260807 08:37:37.158321 18397 webserver.cc:460] Webserver started at http://127.10.207.62:34085/ using document root <none> and password file <none>
I20260807 08:37:37.158663 18397 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:37.158725 18397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:37.158874 18397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:37.159895 18397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "0478ab7382da469eb3e1a239ead25fb6"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "6f71ecf5892176ba8fa270178a84cdb9"
server_key_iv: "8c93f6402db62094148fe107d3af9102"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.160344 18397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "0478ab7382da469eb3e1a239ead25fb6"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "6f71ecf5892176ba8fa270178a84cdb9"
server_key_iv: "8c93f6402db62094148fe107d3af9102"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.161765 18397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:37.162639 18412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:37.162914 18397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:37.163004 18397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "0478ab7382da469eb3e1a239ead25fb6"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "6f71ecf5892176ba8fa270178a84cdb9"
server_key_iv: "8c93f6402db62094148fe107d3af9102"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.163100 18397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:37.177353 18397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:37.177652 18397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:37.177805 18397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:37.182591 18397 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:33849
I20260807 08:37:37.182682 18488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:33849 every 8 connection(s)
I20260807 08:37:37.182991 18397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:37.184135 18489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:37.186843 18489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6: Bootstrap starting.
I20260807 08:37:37.187472 18489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:37.187763 18489 log.cc:824] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:37.188514 18489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6: No bootstrap required, opened a new log
I20260807 08:37:37.190240 18489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0478ab7382da469eb3e1a239ead25fb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33849 } }
I20260807 08:37:37.190392 18489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:37.190415 18489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0478ab7382da469eb3e1a239ead25fb6, State: Initialized, Role: FOLLOWER
I20260807 08:37:37.190591 18489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0478ab7382da469eb3e1a239ead25fb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33849 } }
I20260807 08:37:37.190678 18489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:37.190703 18489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:37.190733 18489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:37.191255 18489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0478ab7382da469eb3e1a239ead25fb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33849 } }
I20260807 08:37:37.191346 18489 leader_election.cc:302] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0478ab7382da469eb3e1a239ead25fb6; no voters:
I20260807 08:37:37.191491 18489 leader_election.cc:288] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:37.191577 18494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:37.191785 18494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 0478ab7382da469eb3e1a239ead25fb6, State: Running, Role: LEADER
I20260807 08:37:37.191843 18489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:37.191916 18494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0478ab7382da469eb3e1a239ead25fb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33849 } }
I20260807 08:37:37.192138 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 18397
I20260807 08:37:37.192257 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:37.192502 11068 external_mini_cluster.cc:1428] Setting key 455bc6dfa30b5c90a5885a3da0aee793
I20260807 08:37:37.192639 18495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0478ab7382da469eb3e1a239ead25fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0478ab7382da469eb3e1a239ead25fb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33849 } } }
I20260807 08:37:37.192687 18496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0478ab7382da469eb3e1a239ead25fb6. Latest consensus state: current_term: 1 leader_uuid: "0478ab7382da469eb3e1a239ead25fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0478ab7382da469eb3e1a239ead25fb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33849 } } }
I20260807 08:37:37.192731 18496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:37.192731 18495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:37.193005 18505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:37.193606 18505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:37.195549 18505 catalog_manager.cc:1347] Generated new cluster ID: ab78f9c5313c4a7a82b384e71cf8fe67
I20260807 08:37:37.195611 18505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:37.200657 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.193300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56214 (local address 127.10.207.62:33849)
0807 08:37:37.194585 (+ 1285us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.194596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.194639 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:37:37.194758 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.194764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.194817 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.195006 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:37.195299 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.196705 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.197911 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.198571 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.198689 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.199768 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:37.199773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:37.199774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:37.200046 (+ 272us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:37.200107 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:37.200126 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:37.200207 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:37.200292 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:37:37.200400 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1156,"thread_start_us":94,"threads_started":1}
I20260807 08:37:37.202114 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:37:37.211066 18505 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:37.211839 18505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:37.221982 18505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0478ab7382da469eb3e1a239ead25fb6: Generated new TSK 0
I20260807 08:37:37.222297 18505 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:37:37.255616 18533 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 08:37:37.255779 18533 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:37:37.256013 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.234718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.10.207.62:33849)
0807 08:37:37.234813 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.234816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.238863 (+ 4047us) server_negotiation.cc:407] Connection header received
0807 08:37:37.238948 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.238949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.238962 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.239041 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:37:37.239253 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.239804 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.241059 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.241251 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.241312 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.241595 (+ 283us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:37:37.255801 (+ 14206us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:37:37.255962 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 08:37:37.256062 18514 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 08:37:37.256603 18532 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:37.234615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:33849 (local address 127.0.0.1:56224)
0807 08:37:37.238705 (+ 4090us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.238720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:37.238839 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:37.239074 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:37.239075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:37.239078 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:37:37.239170 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.239173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.239913 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.239914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.240953 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.240955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.241353 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.241355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.241424 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.241449 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:37:37.256189 (+ 14740us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:37:37.256300 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.207.62:33849: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":4018,"thread_start_us":139,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.207.62:33849: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 08:37:37.257062 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 18397
2026-08-07T08:37:37Z chronyd exiting
I20260807 08:37:37.304260 11068 test_util.cc:175] -----------------------------------------------
I20260807 08:37:37.304330 11068 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091830136701-11068-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (261 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 08:37:37.305124 11068 mini_oidc.cc:152] Starting JWKS server
I20260807 08:37:37.305824 11068 webserver.cc:460] Webserver started at https://127.0.0.1:41921/ using document root <none> and password file <none>
I20260807 08:37:37.305868 11068 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:37:37.306053 11068 webserver.cc:460] Webserver started at http://127.0.0.1:33253/ using document root <none> and password file <none>
2026-08-07T08:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:37Z Disabled control of system clock
I20260807 08:37:37.316246 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:36833
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:36833
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41921/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/testchainca.cert with env {}
W20260807 08:37:37.401989 18540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:37.402237 18540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:37.402276 18540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:37.403491 18540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:37.403558 18540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:37.403584 18540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:37.403617 18540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:37.403644 18540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:37:37.403672 18540 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:37:37.403698 18540 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41921/jwks.json
I20260807 08:37:37.405203 18540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:36833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:36833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41921/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:37.405503 18540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:37.405723 18540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:37.410955 18548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.411001 18550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.411005 18547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:37.411437 18540 server_base.cc:1034] running on GCE node
I20260807 08:37:37.411692 18540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:37.411998 18540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:37.413164 18540 hybrid_clock.cc:648] HybridClock initialized: now 1786091857413141 us; error 39 us; skew 500 ppm
I20260807 08:37:37.414808 18540 webserver.cc:460] Webserver started at http://127.10.207.62:44743/ using document root <none> and password file <none>
I20260807 08:37:37.415073 18540 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:37.415143 18540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:37.415287 18540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:37.416395 18540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "9aba75534be847a8ac76f03658b84df5"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "c64ebad35ef4c3962e9db555d69d13c5"
server_key_iv: "4aafce58ae25ee48c4890a1cd1003358"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.416841 18540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "9aba75534be847a8ac76f03658b84df5"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "c64ebad35ef4c3962e9db555d69d13c5"
server_key_iv: "4aafce58ae25ee48c4890a1cd1003358"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.418433 18540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:37.419370 18559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:37.419631 18540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:37.419742 18540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "9aba75534be847a8ac76f03658b84df5"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "c64ebad35ef4c3962e9db555d69d13c5"
server_key_iv: "4aafce58ae25ee48c4890a1cd1003358"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.419827 18540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:37.457530 18540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:37.457871 18540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:37.458034 18540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:37.462694 18540 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:36833
I20260807 08:37:37.462741 18634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:36833 every 8 connection(s)
I20260807 08:37:37.463075 18540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:37.463897 18635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:37.467525 18635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5: Bootstrap starting.
I20260807 08:37:37.468261 18635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:37.468605 18635 log.cc:824] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:37.469573 18635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5: No bootstrap required, opened a new log
I20260807 08:37:37.471553 18635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aba75534be847a8ac76f03658b84df5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36833 } }
I20260807 08:37:37.471719 18635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:37.471745 18635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aba75534be847a8ac76f03658b84df5, State: Initialized, Role: FOLLOWER
I20260807 08:37:37.471953 18635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aba75534be847a8ac76f03658b84df5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36833 } }
I20260807 08:37:37.472028 18635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:37.472071 18635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:37.472139 18635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:37.472525 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 18540
I20260807 08:37:37.472644 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:37.472795 18635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aba75534be847a8ac76f03658b84df5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36833 } }
I20260807 08:37:37.472853 11068 external_mini_cluster.cc:1428] Setting key ec6490f974dee9bc04b79f7ffcb739ef
I20260807 08:37:37.472900 18635 leader_election.cc:302] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9aba75534be847a8ac76f03658b84df5; no voters:
I20260807 08:37:37.473143 18635 leader_election.cc:288] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:37.473227 18638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:37.473410 18638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [term 1 LEADER]: Becoming Leader. State: Replica: 9aba75534be847a8ac76f03658b84df5, State: Running, Role: LEADER
I20260807 08:37:37.473456 18635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:37.473536 18638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aba75534be847a8ac76f03658b84df5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36833 } }
I20260807 08:37:37.473969 18641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9aba75534be847a8ac76f03658b84df5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aba75534be847a8ac76f03658b84df5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36833 } } }
I20260807 08:37:37.474059 18641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:37.474373 18642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9aba75534be847a8ac76f03658b84df5. Latest consensus state: current_term: 1 leader_uuid: "9aba75534be847a8ac76f03658b84df5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aba75534be847a8ac76f03658b84df5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36833 } } }
I20260807 08:37:37.474547 18642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:37.475323 18654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:37.476191 18654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:37.478058 18654 catalog_manager.cc:1347] Generated new cluster ID: fdd38975cf704371b4586ea90983a323
I20260807 08:37:37.478113 18654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:37.480207 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.473671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.10.207.62:36833)
0807 08:37:37.473890 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.474547 (+ 657us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.474592 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:37:37.474840 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.474849 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.474933 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.475112 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:37.475421 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.476799 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.477830 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.478444 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.478589 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.479476 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:37.479480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:37.479480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:37.479680 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:37.479721 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:37.479735 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:37.479803 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:37.479917 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 08:37:37.479999 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260807 08:37:37.481328 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:37:37.485132 18654 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:37.485632 18654 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:37.489352 18654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9aba75534be847a8ac76f03658b84df5: Generated new TSK 0
I20260807 08:37:37.489635 18654 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:37:37.514938 18683 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 08:37:37.515013 18683 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:37:37.515336 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.500264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.10.207.62:36833)
0807 08:37:37.500309 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.500310 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.500416 (+ 106us) server_negotiation.cc:407] Connection header received
0807 08:37:37.500499 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.500499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.500510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.500566 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:37:37.500742 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.501207 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.502436 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.502730 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.502793 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.503047 (+ 254us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:37:37.515155 (+ 12108us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:37:37.515279 (+ 124us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46104: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 08:37:37.515408 18644 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46104: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 08:37:37.515573 18681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:37.500122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:36833 (local address 127.0.0.1:46104)
0807 08:37:37.500325 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.500338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:37.500419 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:37.500596 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:37.500597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:37.500599 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:37:37.500669 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.500672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.501310 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.501311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.502336 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.502339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.502815 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.502816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.502881 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.502887 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:37:37.515285 (+ 12398us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:37:37.515445 (+ 160us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.207.62:36833: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":131,"threads_started":1}
I20260807 08:37:37.515969 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 18540
2026-08-07T08:37:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 08:37:37.556668 11068 mini_oidc.cc:152] Starting JWKS server
I20260807 08:37:37.557397 11068 webserver.cc:460] Webserver started at https://127.0.0.1:38905/ using document root <none> and password file <none>
I20260807 08:37:37.557473 11068 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:37:37.557649 11068 webserver.cc:460] Webserver started at http://127.0.0.1:34089/ using document root <none> and password file <none>
2026-08-07T08:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:37Z Disabled control of system clock
I20260807 08:37:37.567935 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:46633
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:38189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:46633
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38905/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:37:37.651901 18690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:37.652128 18690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:37.652166 18690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:37.653472 18690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:37.653546 18690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:37.653570 18690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:37.653604 18690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:37.653627 18690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:37:37.653657 18690 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:37:37.653678 18690 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38905/jwks.json
I20260807 08:37:37.655375 18690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:46633
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:46633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38905/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:37.655654 18690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:37.655951 18690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:37.660981 18697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.660975 18695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.660974 18699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:37.661360 18690 server_base.cc:1034] running on GCE node
I20260807 08:37:37.661612 18690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:37.661937 18690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:37.663117 18690 hybrid_clock.cc:648] HybridClock initialized: now 1786091857663092 us; error 43 us; skew 500 ppm
I20260807 08:37:37.664449 18690 webserver.cc:460] Webserver started at http://127.10.207.62:35217/ using document root <none> and password file <none>
I20260807 08:37:37.664702 18690 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:37.664772 18690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:37.664911 18690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:37.666144 18690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "71a018350b804fb89e1f3ca2c6c90441"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "40a9724e20891a0987433901d7aa3bee"
server_key_iv: "5706417350ee8de389203a7f6578bb76"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.666522 18690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "71a018350b804fb89e1f3ca2c6c90441"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "40a9724e20891a0987433901d7aa3bee"
server_key_iv: "5706417350ee8de389203a7f6578bb76"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.668020 18690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:37.668926 18707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:37.669210 18690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:37.669325 18690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "71a018350b804fb89e1f3ca2c6c90441"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "40a9724e20891a0987433901d7aa3bee"
server_key_iv: "5706417350ee8de389203a7f6578bb76"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.669414 18690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:37.686308 18690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:37.686614 18690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:37.686766 18690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:37.691409 18690 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:46633
I20260807 08:37:37.691473 18778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:46633 every 8 connection(s)
I20260807 08:37:37.691830 18690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:37.692766 18779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:37.693992 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 18690
I20260807 08:37:37.694108 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:37.694294 11068 external_mini_cluster.cc:1428] Setting key 6a8358640aa33023ad69132bfd8011c4
I20260807 08:37:37.697389 18779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441: Bootstrap starting.
I20260807 08:37:37.698284 18779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:37.698690 18779 log.cc:824] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:37.699681 18779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441: No bootstrap required, opened a new log
I20260807 08:37:37.701508 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.694905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57578 (local address 127.10.207.62:46633)
0807 08:37:37.695214 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.695226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.695269 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:37:37.695939 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.695950 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.696060 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.696254 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:37.696444 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.698000 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.699105 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.699763 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.699919 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.700649 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:37.700656 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:37.700656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:37.700913 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:37.700963 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:37.700980 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:37.701080 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:37.701156 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 08:37:37.701257 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
I20260807 08:37:37.702512 18779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a018350b804fb89e1f3ca2c6c90441" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46633 } }
I20260807 08:37:37.702729 18779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:37.702785 18779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a018350b804fb89e1f3ca2c6c90441, State: Initialized, Role: FOLLOWER
I20260807 08:37:37.702880 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:37:37.702957 18779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a018350b804fb89e1f3ca2c6c90441" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46633 } }
I20260807 08:37:37.703060 18779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:37.703089 18779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:37.703171 18779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:37.703855 18779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a018350b804fb89e1f3ca2c6c90441" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46633 } }
I20260807 08:37:37.703994 18779 leader_election.cc:302] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71a018350b804fb89e1f3ca2c6c90441; no voters:
I20260807 08:37:37.704237 18779 leader_election.cc:288] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:37.704288 18785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:37.704464 18785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [term 1 LEADER]: Becoming Leader. State: Replica: 71a018350b804fb89e1f3ca2c6c90441, State: Running, Role: LEADER
I20260807 08:37:37.704599 18779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:37.704587 18785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a018350b804fb89e1f3ca2c6c90441" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46633 } }
I20260807 08:37:37.705116 18787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71a018350b804fb89e1f3ca2c6c90441. Latest consensus state: current_term: 1 leader_uuid: "71a018350b804fb89e1f3ca2c6c90441" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a018350b804fb89e1f3ca2c6c90441" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46633 } } }
I20260807 08:37:37.705428 18787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:37.705332 18786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71a018350b804fb89e1f3ca2c6c90441" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a018350b804fb89e1f3ca2c6c90441" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46633 } } }
I20260807 08:37:37.705669 18786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:37.705801 18796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:37.706431 18796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:37.708385 18796 catalog_manager.cc:1347] Generated new cluster ID: cd75dde16a8a45f597610fc4b3d6dd69
I20260807 08:37:37.708448 18796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:37.746877 18796 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:37.747617 18796 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:37.761422 18796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71a018350b804fb89e1f3ca2c6c90441: Generated new TSK 0
I20260807 08:37:37.761590 18796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:37:37.778828 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.775316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57588 (local address 127.10.207.62:46633)
0807 08:37:37.775408 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.775411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.775541 (+ 130us) server_negotiation.cc:407] Connection header received
0807 08:37:37.775643 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.775643 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.775654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.775730 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:37:37.775872 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.776376 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.777181 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.777377 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.777453 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.777653 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:37:37.778608 (+ 955us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38905/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:37:37.778754 (+ 146us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38905/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 08:37:37.778891 18783 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38905/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 08:37:37.780903 18823 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:37.775242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:46633 (local address 127.0.0.1:57588)
0807 08:37:37.775451 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:37.775461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:37.775557 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:37.775731 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:37.775733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:37.775734 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:37:37.775799 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.775801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.776464 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.776465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.777087 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:37.777089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:37.777459 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:37.777461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:37.777526 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:37.777532 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:37:37.778758 (+ 1226us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38905/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:37:37.780776 (+ 2018us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.207.62:46633: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38905/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":1709,"thread_start_us":127,"threads_started":1}
I20260807 08:37:37.781378 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 18690
E20260807 08:37:37.793315 18824 webserver.cc:533] Webserver: sslize failed:
E20260807 08:37:37.793419 18824 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T08:37:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (272 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T08:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:37Z Disabled control of system clock
I20260807 08:37:37.839246 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45079
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:42553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:45079
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/testchainca.cert with env {}
W20260807 08:37:37.933218 18832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:37.933562 18832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:37.933612 18832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:37.935050 18832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:37.935122 18832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:37.935145 18832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:37.935168 18832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:37.935199 18832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:37:37.935225 18832 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:37:37.935250 18832 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 08:37:37.936893 18832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:42553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:45079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:37.937227 18832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:37.937472 18832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:37.943109 18841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.943076 18843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:37.943077 18838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:37.943341 18832 server_base.cc:1034] running on GCE node
I20260807 08:37:37.943614 18832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:37.943894 18832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:37.945056 18832 hybrid_clock.cc:648] HybridClock initialized: now 1786091857945041 us; error 38 us; skew 500 ppm
I20260807 08:37:37.946455 18832 webserver.cc:460] Webserver started at http://127.10.207.62:38179/ using document root <none> and password file <none>
I20260807 08:37:37.946758 18832 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:37.946815 18832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:37.946908 18832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:37.948118 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "a0f1a71899c54f029626866cafa377c5"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "233f46ee5e46278cdf489771f60461ba"
server_key_iv: "59b26fea6506c5ffb4ee39b44fac1fab"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.948482 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "a0f1a71899c54f029626866cafa377c5"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "233f46ee5e46278cdf489771f60461ba"
server_key_iv: "59b26fea6506c5ffb4ee39b44fac1fab"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.950073 18832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:37.950925 18851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:37.951221 18832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:37.951345 18832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "a0f1a71899c54f029626866cafa377c5"
format_stamp: "Formatted at 2026-08-07 08:37:37 on dist-test-slave-h6n0"
server_key: "233f46ee5e46278cdf489771f60461ba"
server_key_iv: "59b26fea6506c5ffb4ee39b44fac1fab"
server_key_version: "encryptionkey@0"
I20260807 08:37:37.951424 18832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:37.983944 18832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:37.984258 18832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:37.984421 18832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:37.988871 18832 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:45079
I20260807 08:37:37.989001 18924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:45079 every 8 connection(s)
I20260807 08:37:37.989307 18832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:37.990495 18926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:37.993314 18926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5: Bootstrap starting.
I20260807 08:37:37.994046 18926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:37.994367 18926 log.cc:824] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:37.995319 18926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5: No bootstrap required, opened a new log
I20260807 08:37:37.995965 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 18832
I20260807 08:37:37.996090 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:37.996271 11068 external_mini_cluster.cc:1428] Setting key 09156cc4746c0da6f562bd5bdc2e4b90
I20260807 08:37:37.997769 18926 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f1a71899c54f029626866cafa377c5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45079 } }
I20260807 08:37:37.997979 18926 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:37.998025 18926 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f1a71899c54f029626866cafa377c5, State: Initialized, Role: FOLLOWER
I20260807 08:37:37.998206 18926 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f1a71899c54f029626866cafa377c5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45079 } }
I20260807 08:37:37.998312 18926 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:37.998350 18926 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:37.998409 18926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:37.999343 18926 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f1a71899c54f029626866cafa377c5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45079 } }
I20260807 08:37:37.999473 18926 leader_election.cc:302] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0f1a71899c54f029626866cafa377c5; no voters:
I20260807 08:37:37.999663 18926 leader_election.cc:288] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:37.999785 18932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:37.999949 18926 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:38.000012 18932 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [term 1 LEADER]: Becoming Leader. State: Replica: a0f1a71899c54f029626866cafa377c5, State: Running, Role: LEADER
I20260807 08:37:38.000139 18932 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f1a71899c54f029626866cafa377c5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45079 } }
I20260807 08:37:38.000532 18932 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0f1a71899c54f029626866cafa377c5. Latest consensus state: current_term: 1 leader_uuid: "a0f1a71899c54f029626866cafa377c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f1a71899c54f029626866cafa377c5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45079 } } }
I20260807 08:37:38.000643 18932 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:38.000548 18934 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0f1a71899c54f029626866cafa377c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f1a71899c54f029626866cafa377c5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45079 } } }
I20260807 08:37:38.000739 18934 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:38.001577 18942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:38.002190 18942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:38.004221 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:37.996842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.10.207.62:45079)
0807 08:37:37.997128 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:37.997142 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:37.997185 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:37:37.997720 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:37.997730 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:37.997833 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:37.998044 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:37.998270 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:38.000206 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:38.001427 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:38.002156 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:38.002291 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:38.003347 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:38.003351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:38.003352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:37:38.003578 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:38.003618 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:37:38.003630 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:38.003720 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:38.003881 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 08:37:38.004000 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260807 08:37:38.004262 18942 catalog_manager.cc:1347] Generated new cluster ID: ac3efbfa27504ba7b60f5e039b881b32
I20260807 08:37:38.004338 18942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:38.005468 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:37:38.024098 18942 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:38.024902 18942 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:38.034484 18942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0f1a71899c54f029626866cafa377c5: Generated new TSK 0
I20260807 08:37:38.034853 18942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:37:38.900287 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:38.044430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.10.207.62:45079)
0807 08:37:38.044485 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:38.044488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:38.044585 (+ 97us) server_negotiation.cc:407] Connection header received
0807 08:37:38.044663 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:38.044664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:38.044678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:38.044761 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:37:38.044904 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:38.045678 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:38.046556 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:38.046867 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:38.046949 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:38.047133 (+ 184us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:37:38.899988 (+852855us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:37:38.900205 (+ 217us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35648: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":29}
W20260807 08:37:38.900373 18931 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35648: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 08:37:38.900456 18964 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:38.044333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45079 (local address 127.0.0.1:35648)
0807 08:37:38.044486 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:38.044498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:38.044576 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:38.044768 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:38.044769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:38.044772 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:37:38.044829 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:38.044831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:38.045792 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:38.045792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:38.046434 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:38.046436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:38.046963 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:38.046964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:38.047012 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:38.047018 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:37:38.900173 (+853155us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:37:38.900316 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.207.62:45079: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260807 08:37:38.900978 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 18832
2026-08-07T08:37:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1096 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (99 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (94 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:39 dist-test-slave-h6n0 krb5kdc[18977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:39 dist-test-slave-h6n0 krb5kdc[18977](info): set up 2 sockets
Aug 07 08:37:39 dist-test-slave-h6n0 krb5kdc[18977](info): commencing operation
krb5kdc: starting...
W20260807 08:37:40.369866 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:40 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:40.513106 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:39509
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:39509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:40.607694 18993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:40.607977 18993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:40.608006 18993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:40.609504 18993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:40.609555 18993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:40.609571 18993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:40.609583 18993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:40.609596 18993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:40.611286 18993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:39509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:39509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:40.611624 18993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:40.611917 18993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:40.617790 18999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:40.617834 19000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:40.617834 19005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:40.617957 18993 server_base.cc:1034] running on GCE node
I20260807 08:37:40.618470 18993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:40.618907 18993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:40.620061 18993 hybrid_clock.cc:648] HybridClock initialized: now 1786091860620036 us; error 49 us; skew 500 ppm
Aug 07 08:37:40 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091860, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:40.623018 18993 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:40.623564 18993 webserver.cc:460] Webserver started at http://127.10.207.62:35585/ using document root <none> and password file <none>
I20260807 08:37:40.623788 18993 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:40.623834 18993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:40.623937 18993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:40.625154 18993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "d520b658424b469ebd11b44a5fec3cb6"
format_stamp: "Formatted at 2026-08-07 08:37:40 on dist-test-slave-h6n0"
server_key: "86c88d4f01a9eff55d004684fb107051"
server_key_iv: "0512df9c63d68a43b9c66aff1b9edf04"
server_key_version: "encryptionkey@0"
I20260807 08:37:40.625517 18993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "d520b658424b469ebd11b44a5fec3cb6"
format_stamp: "Formatted at 2026-08-07 08:37:40 on dist-test-slave-h6n0"
server_key: "86c88d4f01a9eff55d004684fb107051"
server_key_iv: "0512df9c63d68a43b9c66aff1b9edf04"
server_key_version: "encryptionkey@0"
I20260807 08:37:40.627429 18993 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 08:37:40.628300 19014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:40.628567 18993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:40.628684 18993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "d520b658424b469ebd11b44a5fec3cb6"
format_stamp: "Formatted at 2026-08-07 08:37:40 on dist-test-slave-h6n0"
server_key: "86c88d4f01a9eff55d004684fb107051"
server_key_iv: "0512df9c63d68a43b9c66aff1b9edf04"
server_key_version: "encryptionkey@0"
I20260807 08:37:40.628774 18993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:40.648099 18993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:40.650490 18993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:40.650738 18993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:40.656069 18993 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:39509
I20260807 08:37:40.656136 19082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:39509 every 8 connection(s)
I20260807 08:37:40.656584 18993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:40.657629 19083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:40.660174 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 18993
I20260807 08:37:40.660322 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:40.660274 19083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6: Bootstrap starting.
I20260807 08:37:40.660557 11068 external_mini_cluster.cc:1428] Setting key ace2a7652b83c5df772a6caed13a5a7b
I20260807 08:37:40.661078 19083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:40.661489 19083 log.cc:824] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:40.662336 19083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6: No bootstrap required, opened a new log
I20260807 08:37:40.664960 19083 raft_consensus.cc:348] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520b658424b469ebd11b44a5fec3cb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39509 } }
I20260807 08:37:40.665138 19083 raft_consensus.cc:374] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:40.665172 19083 raft_consensus.cc:729] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d520b658424b469ebd11b44a5fec3cb6, State: Initialized, Role: FOLLOWER
I20260807 08:37:40.665294 19083 consensus_queue.cc:260] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520b658424b469ebd11b44a5fec3cb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39509 } }
I20260807 08:37:40.665365 19083 raft_consensus.cc:388] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:40.665395 19083 raft_consensus.cc:482] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:40.665448 19083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:40.666066 19083 raft_consensus.cc:504] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520b658424b469ebd11b44a5fec3cb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39509 } }
I20260807 08:37:40.666188 19083 leader_election.cc:302] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d520b658424b469ebd11b44a5fec3cb6; no voters:
I20260807 08:37:40.666379 19083 leader_election.cc:288] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:40.666450 19090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:40.666728 19083 sys_catalog.cc:564] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:40.666815 19090 raft_consensus.cc:686] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [term 1 LEADER]: Becoming Leader. State: Replica: d520b658424b469ebd11b44a5fec3cb6, State: Running, Role: LEADER
I20260807 08:37:40.666973 19090 consensus_queue.cc:237] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520b658424b469ebd11b44a5fec3cb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39509 } }
I20260807 08:37:40.667589 19091 sys_catalog.cc:455] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d520b658424b469ebd11b44a5fec3cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520b658424b469ebd11b44a5fec3cb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39509 } } }
I20260807 08:37:40.667691 19091 sys_catalog.cc:458] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:40.667598 19092 sys_catalog.cc:455] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d520b658424b469ebd11b44a5fec3cb6. Latest consensus state: current_term: 1 leader_uuid: "d520b658424b469ebd11b44a5fec3cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d520b658424b469ebd11b44a5fec3cb6" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39509 } } }
I20260807 08:37:40.667951 19092 sys_catalog.cc:458] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:40.668264 19099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 08:37:40 dist-test-slave-h6n0 krb5kdc[18977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:40.668951 19099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:40.670972 19099 catalog_manager.cc:1347] Generated new cluster ID: 9b7ff891fc61482bbcdc6b9150e65e5f
I20260807 08:37:40.671082 19099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:40.673233 19088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:40.661217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.10.207.62:39509)
0807 08:37:40.661490 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:40.661500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:40.661532 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:37:40.662280 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:40.662289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:40.662359 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:40.662624 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:40.663445 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:40.664421 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:40.665517 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:40.665991 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:40.666084 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:40.669312 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:40.669317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:40.669317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:40.669333 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:40.671543 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:40.671944 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:40.671948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:40.671949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:40.672049 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:40.672289 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:40.672291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:40.672291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:40.672565 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:40.672785 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:40.672870 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:37:40.673028 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":135,"threads_started":1}
I20260807 08:37:40.685791 19099 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:40.686421 19099 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:40.690034 19099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d520b658424b469ebd11b44a5fec3cb6: Generated new TSK 0
I20260807 08:37:40.690255 19099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:40.721988 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:39509
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:40.836946 19114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:40.837361 19114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:40.837436 19114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:40.839249 19114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:40.839362 19114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:40.839438 19114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:40.841753 19114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:39509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:40.842216 19114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:40.842593 19114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:40.848796 19122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:40.848898 19120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:40.848975 19119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:40.849258 19114 server_base.cc:1034] running on GCE node
I20260807 08:37:40.849541 19114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:40.849865 19114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:40.851058 19114 hybrid_clock.cc:648] HybridClock initialized: now 1786091860851035 us; error 51 us; skew 500 ppm
Aug 07 08:37:40 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091860, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:40.854015 19114 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:40.854514 19114 webserver.cc:460] Webserver started at http://127.10.207.1:36139/ using document root <none> and password file <none>
I20260807 08:37:40.854913 19114 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:40.854969 19114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:40.855106 19114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:40.856230 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "2c628cd4983a49d8ad5d69c69854576f"
format_stamp: "Formatted at 2026-08-07 08:37:40 on dist-test-slave-h6n0"
server_key: "959208cae0e78f3f8ffece1dd81de668"
server_key_iv: "9283eaf896c4baf294838f6596360f0e"
server_key_version: "encryptionkey@0"
I20260807 08:37:40.856607 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "2c628cd4983a49d8ad5d69c69854576f"
format_stamp: "Formatted at 2026-08-07 08:37:40 on dist-test-slave-h6n0"
server_key: "959208cae0e78f3f8ffece1dd81de668"
server_key_iv: "9283eaf896c4baf294838f6596360f0e"
server_key_version: "encryptionkey@0"
I20260807 08:37:40.858181 19114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:40.859169 19133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:40.859467 19114 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:37:40.859580 19114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "2c628cd4983a49d8ad5d69c69854576f"
format_stamp: "Formatted at 2026-08-07 08:37:40 on dist-test-slave-h6n0"
server_key: "959208cae0e78f3f8ffece1dd81de668"
server_key_iv: "9283eaf896c4baf294838f6596360f0e"
server_key_version: "encryptionkey@0"
I20260807 08:37:40.859663 19114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:40.874493 19114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:40.876641 19114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:40.876801 19114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:40.877064 19114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:40.877403 19114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:40.877471 19114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:40.877513 19114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:40.877542 19114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:40.884754 19114 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:37959
I20260807 08:37:40.884931 19289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:37959 every 8 connection(s)
I20260807 08:37:40.885177 19114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:40.890605 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 19114
I20260807 08:37:40.890743 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:40.890949 11068 external_mini_cluster.cc:1428] Setting key bfb822e0cacda515a5d4e437f237cc42
Aug 07 08:37:40 dist-test-slave-h6n0 krb5kdc[18977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091860, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:40.896628 19088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:40.886056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:51935 (local address 127.10.207.62:39509)
0807 08:37:40.886152 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:40.886155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:40.886384 (+ 229us) server_negotiation.cc:407] Connection header received
0807 08:37:40.887558 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:40.887559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:40.887575 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:40.887643 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:40.888430 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:40.888986 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:40.890077 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:40.890307 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:40.890400 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:40.893347 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:40.893348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:40.893349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:40.893358 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:40.894999 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:40.895489 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:40.895491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:40.895491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:40.895562 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:40.895855 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:40.895855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:40.895856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:40.896003 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:40.896080 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:40.896406 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 08:37:40.896519 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:37:40.896878 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:40.885960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:39509 (local address 127.10.207.1:51935)
0807 08:37:40.886207 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:40.886266 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:40.887414 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:40.887731 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:40.887736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:40.888071 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:40.888305 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:40.888310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:40.889087 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:40.889089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:40.889979 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:40.889981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:40.890416 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:40.890417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:40.890644 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:40.891394 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:40.891405 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:40.893216 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:40.895111 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:40.895121 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:40.895124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:40.895392 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:40.895635 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:40.895636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:40.895637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:40.895774 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:40.896081 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:40.896091 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:40.896259 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 08:37:40.896384 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 08:37:40.896581 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":102,"threads_started":1}
I20260807 08:37:40.897496 19290 heartbeater.cc:347] Connected to a master server at 127.10.207.62:39509
I20260807 08:37:40.897645 19290 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:40.897872 19290 heartbeater.cc:511] Master 127.10.207.62:39509 requested a full tablet report, sending...
I20260807 08:37:40.898689 19039 ts_manager.cc:194] Registered new tserver with Master: 2c628cd4983a49d8ad5d69c69854576f (127.10.207.1:37959)
I20260807 08:37:40.899746 19039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:51935
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:40.921928 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:39509
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:41.014266 19295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:41.014514 19295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:41.014562 19295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:41.015758 19295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:41.015803 19295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:41.015831 19295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:41.017402 19295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:39509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:41.017689 19295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:41.017978 19295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:41.023797 19304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:41.023815 19303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:41.023957 19306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:41.024168 19295 server_base.cc:1034] running on GCE node
I20260807 08:37:41.024410 19295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:41.024636 19295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:41.025796 19295 hybrid_clock.cc:648] HybridClock initialized: now 1786091861025783 us; error 32 us; skew 500 ppm
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.029078 19295 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:41.029565 19295 webserver.cc:460] Webserver started at http://127.10.207.2:40389/ using document root <none> and password file <none>
I20260807 08:37:41.029789 19295 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:41.029837 19295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:41.030042 19295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:41.031332 19295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "a6fc5b7d31264e2f934e3237e16386de"
format_stamp: "Formatted at 2026-08-07 08:37:41 on dist-test-slave-h6n0"
server_key: "91af4b859c3dcfce161f894220f09dc2"
server_key_iv: "fb0f4b9a9958b780d55ddfe0fe9b8714"
server_key_version: "encryptionkey@0"
I20260807 08:37:41.031800 19295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "a6fc5b7d31264e2f934e3237e16386de"
format_stamp: "Formatted at 2026-08-07 08:37:41 on dist-test-slave-h6n0"
server_key: "91af4b859c3dcfce161f894220f09dc2"
server_key_iv: "fb0f4b9a9958b780d55ddfe0fe9b8714"
server_key_version: "encryptionkey@0"
I20260807 08:37:41.033386 19295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:37:41.034229 19315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:41.034507 19295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:41.034683 19295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "a6fc5b7d31264e2f934e3237e16386de"
format_stamp: "Formatted at 2026-08-07 08:37:41 on dist-test-slave-h6n0"
server_key: "91af4b859c3dcfce161f894220f09dc2"
server_key_iv: "fb0f4b9a9958b780d55ddfe0fe9b8714"
server_key_version: "encryptionkey@0"
I20260807 08:37:41.034792 19295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:41.055182 19295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:41.058175 19295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:41.058319 19295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:41.058652 19295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:41.059078 19295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:41.059118 19295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:41.059151 19295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:41.059206 19295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:41.067004 19295 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:40979
I20260807 08:37:41.067130 19477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:40979 every 8 connection(s)
I20260807 08:37:41.067548 19295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:41.068681 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 19295
I20260807 08:37:41.068807 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:41.069036 11068 external_mini_cluster.cc:1428] Setting key bb8561afb617e5e43c35a3680adab7e8
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:41.078985 19088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.068469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:54401 (local address 127.10.207.62:39509)
0807 08:37:41.068574 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.068578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.068889 (+ 311us) server_negotiation.cc:407] Connection header received
0807 08:37:41.070275 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.070276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.070287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.070345 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.071321 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.071919 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.072964 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.073295 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.073371 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:41.075871 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:41.075875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:41.075875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:41.075890 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:41.077438 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.077859 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.077860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.077860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.077926 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.078195 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.078196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.078196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.078340 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:41.078425 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:41.078780 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 08:37:41.078892 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:37:41.079185 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.068469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:39509 (local address 127.10.207.2:54401)
0807 08:37:41.068711 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:41.068769 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:41.070156 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:41.070445 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:41.070449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:41.070903 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:41.071214 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:41.071219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.072035 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.072036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:41.072869 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:41.072871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.073396 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.073397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:41.073508 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:41.074133 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:41.074143 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:41.075744 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:41.077544 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:41.077551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:41.077554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:41.077775 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:41.077998 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:41.077999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:41.077999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:41.078116 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:41.078419 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:41.078427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:41.078625 (+ 198us) client_negotiation.cc:769] Sending connection context
0807 08:37:41.078739 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 08:37:41.078933 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
I20260807 08:37:41.079836 19479 heartbeater.cc:347] Connected to a master server at 127.10.207.62:39509
I20260807 08:37:41.079963 19479 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:41.080171 19479 heartbeater.cc:511] Master 127.10.207.62:39509 requested a full tablet report, sending...
I20260807 08:37:41.080629 19039 ts_manager.cc:194] Registered new tserver with Master: a6fc5b7d31264e2f934e3237e16386de (127.10.207.2:40979)
I20260807 08:37:41.081122 19039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:54401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:41.101739 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:39509
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:41.197479 19485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:41.197750 19485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:41.197780 19485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:41.199156 19485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:41.199211 19485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:41.199242 19485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:41.200919 19485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:39509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:41.201236 19485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:41.201568 19485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:41.207471 19496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:41.207463 19492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:41.207463 19493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:41.207813 19485 server_base.cc:1034] running on GCE node
I20260807 08:37:41.208104 19485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:41.208465 19485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:41.209666 19485 hybrid_clock.cc:648] HybridClock initialized: now 1786091861209645 us; error 52 us; skew 500 ppm
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.212474 19485 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:41.212910 19485 webserver.cc:460] Webserver started at http://127.10.207.3:38061/ using document root <none> and password file <none>
I20260807 08:37:41.213125 19485 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:41.213173 19485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:41.213271 19485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:41.214381 19485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "1209aa6653db47ffa21555fdcbf04596"
format_stamp: "Formatted at 2026-08-07 08:37:41 on dist-test-slave-h6n0"
server_key: "98ed4ce83c7f82ba1398a5334c144a3e"
server_key_iv: "751a3cce348dc0232619751a2fb46727"
server_key_version: "encryptionkey@0"
I20260807 08:37:41.214784 19485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "1209aa6653db47ffa21555fdcbf04596"
format_stamp: "Formatted at 2026-08-07 08:37:41 on dist-test-slave-h6n0"
server_key: "98ed4ce83c7f82ba1398a5334c144a3e"
server_key_iv: "751a3cce348dc0232619751a2fb46727"
server_key_version: "encryptionkey@0"
I20260807 08:37:41.216197 19485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:41.217103 19507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:41.217358 19485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:41.217484 19485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "1209aa6653db47ffa21555fdcbf04596"
format_stamp: "Formatted at 2026-08-07 08:37:41 on dist-test-slave-h6n0"
server_key: "98ed4ce83c7f82ba1398a5334c144a3e"
server_key_iv: "751a3cce348dc0232619751a2fb46727"
server_key_version: "encryptionkey@0"
I20260807 08:37:41.217576 19485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:41.247308 19485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:41.249210 19485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:41.249370 19485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:41.249629 19485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:41.249976 19485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:41.250037 19485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:41.250074 19485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:41.250102 19485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:41.257225 19485 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34693
I20260807 08:37:41.257257 19672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34693 every 8 connection(s)
I20260807 08:37:41.257660 19485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:41.259306 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 19485
I20260807 08:37:41.259413 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:41.259620 11068 external_mini_cluster.cc:1428] Setting key b2c766c21655a89039b28f19663e6014
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:41.269421 19088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.258656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:47463 (local address 127.10.207.62:39509)
0807 08:37:41.258703 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.258705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.258970 (+ 265us) server_negotiation.cc:407] Connection header received
0807 08:37:41.260480 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.260482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.260499 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.260572 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.261460 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.262000 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.263239 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.263573 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.263644 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:41.266003 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:41.266004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:41.266004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:41.266011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:41.267922 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.268374 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.268374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.268375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.268431 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.268694 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.268696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.268696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.268830 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:41.268903 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:41.269284 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 08:37:41.269355 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:37:41.269721 19676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.258615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:39509 (local address 127.10.207.3:47463)
0807 08:37:41.258813 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:41.258860 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:41.260329 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:41.260663 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:41.260667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:41.261026 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:41.261316 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:41.261321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.262109 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.262110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:41.263127 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:41.263130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.263663 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.263664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:41.263785 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:41.264367 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:41.264377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:41.265890 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:41.268011 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:41.268020 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:41.268027 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:41.268298 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:41.268499 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:41.268499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:41.268500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:41.268605 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:41.268942 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:41.268949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:41.269119 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 08:37:41.269234 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 08:37:41.269395 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260807 08:37:41.270391 19674 heartbeater.cc:347] Connected to a master server at 127.10.207.62:39509
I20260807 08:37:41.270501 19674 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:41.270768 19674 heartbeater.cc:511] Master 127.10.207.62:39509 requested a full tablet report, sending...
I20260807 08:37:41.271307 19039 ts_manager.cc:194] Registered new tserver with Master: 1209aa6653db47ffa21555fdcbf04596 (127.10.207.3:34693)
I20260807 08:37:41.271776 19039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:47463
I20260807 08:37:41.272197 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/test-admin.keytab.
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.289362 11068 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 08:37:41.291263 19688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:41.290404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.10.207.1:37959)
0807 08:37:41.290619 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.290624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.290637 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:37:41.290663 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.290665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.290679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.290754 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.291199 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37780: BlockingRecv error: recv got EOF from 127.0.0.1:37780 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
W20260807 08:37:41.291414 11068 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.292548 11068 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.293498 11068 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:41.303138 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.294117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.10.207.1:37959)
0807 08:37:41.294175 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.294182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.294195 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:37:41.294271 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.294271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.294281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.294334 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.294870 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.295606 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.296404 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.296721 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.296799 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:41.299606 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:41.299621 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:41.299621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:41.299629 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:41.301868 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.302264 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.302267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.302267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.302335 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.302615 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.302616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.302616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.302765 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:41.302880 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:41.302977 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 08:37:41.303064 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 08:37:41.305734 19688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:41.305029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.10.207.1:37959)
0807 08:37:41.305126 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.305128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.305145 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:37:41.305235 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.305236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.305248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.305310 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.305648 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37804: BlockingRecv error: recv got EOF from 127.0.0.1:37804 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 08:37:41.306188 11068 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.307524 11068 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.308511 11068 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:41.315920 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.309157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.10.207.1:37959)
0807 08:37:41.309245 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.309247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.309258 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:41.309323 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.309324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.309333 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.309385 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.309771 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.310261 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.311137 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.311357 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.311431 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:41.313474 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:41.313475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:41.313475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:41.313483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:41.314822 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.315161 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.315162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.315163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.315221 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.315422 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.315423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.315424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.315520 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:41.315619 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:41.315736 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 08:37:41.315812 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 08:37:41.317648 19688 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:41.317158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.10.207.1:37959)
0807 08:37:41.317234 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.317237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.317262 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:37:41.317281 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.317281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.317290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.317335 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.317606 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37818: BlockingRecv error: recv got EOF from 127.0.0.1:37818 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 08:37:41.317843 11068 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091830136701-11068-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.318945 11068 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:41.319869 11068 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[18977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:37:41.327209 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:41.320523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.10.207.1:37959)
0807 08:37:41.320612 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:41.320615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:41.320662 (+ 47us) server_negotiation.cc:407] Connection header received
0807 08:37:41.320743 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:41.320744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:41.320753 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:41.320821 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:41.321237 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.321785 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.322493 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:41.322741 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:41.322828 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:41.324945 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:41.324946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:41.324946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:41.324952 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:41.326164 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.326494 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.326495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.326495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.326593 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:41.326793 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:41.326794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:41.326794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:41.326888 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:41.326963 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:41.327083 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:37:41.327146 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:37:41.328116 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 19114
I20260807 08:37:41.345517 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 19295
I20260807 08:37:41.353349 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 19485
I20260807 08:37:41.367352 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 18993
2026-08-07T08:37:41Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2273 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[19710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[19710](info): set up 2 sockets
Aug 07 08:37:41 dist-test-slave-h6n0 krb5kdc[19710](info): commencing operation
krb5kdc: starting...
W20260807 08:37:42.643193 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:42 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab.
Entry for principal oryx/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab.
I20260807 08:37:42.782032 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35169
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab
--principal=oryx/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:35169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:42.878294 19729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:42.878610 19729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:42.878677 19729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:42.879982 19729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:42.880098 19729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:42.880141 19729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:42.880173 19729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:42.880203 19729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:42.882268 19729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:35169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab
--principal=oryx/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:35169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:42.882689 19729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:42.882982 19729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:42.889925 19738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:42.890239 19742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:42.890406 19739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:42.890890 19729 server_base.cc:1034] running on GCE node
I20260807 08:37:42.891271 19729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:42.891716 19729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:42.892892 19729 hybrid_clock.cc:648] HybridClock initialized: now 1786091862892870 us; error 38 us; skew 500 ppm
Aug 07 08:37:42 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091862, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:42.896579 19729 init.cc:377] Logged in from keytab as oryx/127.10.207.62@KRBTEST.COM (short username oryx)
I20260807 08:37:42.897123 19729 webserver.cc:460] Webserver started at http://127.10.207.62:39647/ using document root <none> and password file <none>
I20260807 08:37:42.897444 19729 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:42.897517 19729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:42.897680 19729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:42.898926 19729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "9f518004fae84e6b951b3255e3e546a5"
format_stamp: "Formatted at 2026-08-07 08:37:42 on dist-test-slave-h6n0"
server_key: "f36ed36f82637a31929cac6deabf0c3f"
server_key_iv: "dd2104f1912515dc0cd9c28e8614e616"
server_key_version: "encryptionkey@0"
I20260807 08:37:42.899322 19729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "9f518004fae84e6b951b3255e3e546a5"
format_stamp: "Formatted at 2026-08-07 08:37:42 on dist-test-slave-h6n0"
server_key: "f36ed36f82637a31929cac6deabf0c3f"
server_key_iv: "dd2104f1912515dc0cd9c28e8614e616"
server_key_version: "encryptionkey@0"
I20260807 08:37:42.900947 19729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:42.902067 19751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:42.902379 19729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:42.902530 19729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "9f518004fae84e6b951b3255e3e546a5"
format_stamp: "Formatted at 2026-08-07 08:37:42 on dist-test-slave-h6n0"
server_key: "f36ed36f82637a31929cac6deabf0c3f"
server_key_iv: "dd2104f1912515dc0cd9c28e8614e616"
server_key_version: "encryptionkey@0"
I20260807 08:37:42.902678 19729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:42.927765 19729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:42.931033 19729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:42.931247 19729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:42.936491 19729 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:35169
I20260807 08:37:42.936529 19822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:35169 every 8 connection(s)
I20260807 08:37:42.936983 19729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:42.938227 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 19729
I20260807 08:37:42.938416 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:42.938575 19823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:42.938750 11068 external_mini_cluster.cc:1428] Setting key d944f945a849501bb8b68647c0952615
I20260807 08:37:42.943228 19823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: Bootstrap starting.
I20260807 08:37:42.944181 19823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:42.944602 19823 log.cc:824] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:42.945675 19823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: No bootstrap required, opened a new log
Aug 07 08:37:42 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:42.948972 19823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } }
I20260807 08:37:42.949234 19823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:42.949275 19823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f518004fae84e6b951b3255e3e546a5, State: Initialized, Role: FOLLOWER
I20260807 08:37:42.949512 19823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } }
I20260807 08:37:42.949623 19823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:42.949666 19823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:42.949753 19823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:42.950819 19823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } }
I20260807 08:37:42.951154 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:42.939466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.10.207.62:35169)
0807 08:37:42.939699 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:42.939712 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:42.939806 (+ 94us) server_negotiation.cc:407] Connection header received
0807 08:37:42.940584 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:42.940593 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:42.940652 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:42.940842 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:42.941577 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:42.942651 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:42.943518 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:42.943985 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:42.944135 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:42.947345 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:42.947351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:42.947351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:42.947390 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:42.949370 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:42.949811 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:42.949815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:42.949815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:42.949932 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:42.950152 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:42.950153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:42.950153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:42.950445 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:42.950651 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:42.950728 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 08:37:42.950883 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":109,"threads_started":1}
I20260807 08:37:42.951131 19823 leader_election.cc:302] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f518004fae84e6b951b3255e3e546a5; no voters:
I20260807 08:37:42.951445 19823 leader_election.cc:288] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:42.951530 19832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:42.951834 19832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 1 LEADER]: Becoming Leader. State: Replica: 9f518004fae84e6b951b3255e3e546a5, State: Running, Role: LEADER
I20260807 08:37:42.951915 19823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:42.952003 19832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } }
I20260807 08:37:42.952490 19832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f518004fae84e6b951b3255e3e546a5. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } }
I20260807 08:37:42.952503 19833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } }
I20260807 08:37:42.952658 19832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:42.952778 19833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:42.953613 19847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:42.954154 19847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:42.956015 19847 catalog_manager.cc:1347] Generated new cluster ID: 267afb8bff0b4f77ad061cf56a20e1ec
I20260807 08:37:42.956073 19847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:42.973951 19847 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:42.974713 19847 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:42.987023 19847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: Generated new TSK 0
I20260807 08:37:42.987210 19847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab.
Entry for principal oryx/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab.
I20260807 08:37:43.019569 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35169
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab
--principal=oryx/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:43.113933 19868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:43.114210 19868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:43.114248 19868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:43.115415 19868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:43.115485 19868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:43.115533 19868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:43.117072 19868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab
--principal=oryx/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:43.117367 19868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:43.117630 19868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:43.123613 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.123613 19876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:43.123842 19868 server_base.cc:1034] running on GCE node
W20260807 08:37:43.123970 19884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:43.124285 19868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:43.124516 19868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:43.125689 19868 hybrid_clock.cc:648] HybridClock initialized: now 1786091863125680 us; error 36 us; skew 500 ppm
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:43.128865 19868 init.cc:377] Logged in from keytab as oryx/127.10.207.1@KRBTEST.COM (short username oryx)
I20260807 08:37:43.129448 19868 webserver.cc:460] Webserver started at http://127.10.207.1:35023/ using document root <none> and password file <none>
I20260807 08:37:43.129711 19868 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:43.129786 19868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:43.129951 19868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:43.131309 19868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "9bb5534a84574f759c0644363266b675"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "9a3f5f4ba77a2eac29aad817182ade8a"
server_key_iv: "bd7d0eee5d16942e8605467ff41585b3"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.131727 19868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "9bb5534a84574f759c0644363266b675"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "9a3f5f4ba77a2eac29aad817182ade8a"
server_key_iv: "bd7d0eee5d16942e8605467ff41585b3"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.133522 19868 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:37:43.134639 19893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.134881 19868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:43.135000 19868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "9bb5534a84574f759c0644363266b675"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "9a3f5f4ba77a2eac29aad817182ade8a"
server_key_iv: "bd7d0eee5d16942e8605467ff41585b3"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.135107 19868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:43.145381 19868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:43.147943 19868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:43.148140 19868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:43.148432 19868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:43.148821 19868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:43.148880 19868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.148927 19868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:43.148953 19868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.157114 19868 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:42489
I20260807 08:37:43.157234 20053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:42489 every 8 connection(s)
I20260807 08:37:43.157629 19868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:43.166868 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 19868
I20260807 08:37:43.166977 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:43.167232 11068 external_mini_cluster.cc:1428] Setting key b01575618d5004860380f23d3200f4a0
I20260807 08:37:43.168716 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.158544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:40029 (local address 127.10.207.62:35169)
0807 08:37:43.158653 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.158656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.158890 (+ 234us) server_negotiation.cc:407] Connection header received
0807 08:37:43.159564 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.159565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.159583 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.159653 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.160746 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.161312 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.162457 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.162780 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.162843 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.165246 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.165247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.165247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.165254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.166769 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.167349 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.167350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.167350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.167417 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.167795 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.167796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.167796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.167989 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.168079 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.168443 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.168580 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:37:43.168761 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.158465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.1:40029)
0807 08:37:43.158722 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.158768 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.159418 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.159720 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.159724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.160376 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.160625 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.160630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.161419 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.161420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.162362 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.162365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.162867 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.162868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.162984 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.163535 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.163544 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.165111 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.166865 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.166871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.166874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.167236 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.167520 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.167522 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.167523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.167706 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.168090 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.168099 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.168275 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.168378 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.168561 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":163,"threads_started":1}
I20260807 08:37:43.169536 20054 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35169
I20260807 08:37:43.169654 20054 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:43.169857 20054 heartbeater.cc:511] Master 127.10.207.62:35169 requested a full tablet report, sending...
I20260807 08:37:43.170560 19775 ts_manager.cc:194] Registered new tserver with Master: 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489)
I20260807 08:37:43.171458 19775 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:40029
WARNING: no policy specified for oryx/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab.
Entry for principal oryx/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab.
I20260807 08:37:43.197728 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35169
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab
--principal=oryx/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:43.293435 20060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:43.293680 20060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:43.293722 20060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:43.294955 20060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:43.295003 20060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:43.295030 20060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:43.296479 20060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab
--principal=oryx/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:43.296710 20060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:43.296959 20060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:43.302964 20072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.302977 20065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.302981 20067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:43.303349 20060 server_base.cc:1034] running on GCE node
I20260807 08:37:43.303574 20060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:43.303826 20060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:43.305068 20060 hybrid_clock.cc:648] HybridClock initialized: now 1786091863305027 us; error 54 us; skew 500 ppm
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:43.308440 20060 init.cc:377] Logged in from keytab as oryx/127.10.207.2@KRBTEST.COM (short username oryx)
I20260807 08:37:43.309048 20060 webserver.cc:460] Webserver started at http://127.10.207.2:38479/ using document root <none> and password file <none>
I20260807 08:37:43.309309 20060 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:43.309386 20060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:43.309535 20060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:43.310696 20060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "b60b419870264b5997888887b3177cac"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "55d4bce34c92bb7afba29fdb524ad650"
server_key_iv: "748bf3f5ecee61af20901338fa7133d4"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.311100 20060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "b60b419870264b5997888887b3177cac"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "55d4bce34c92bb7afba29fdb524ad650"
server_key_iv: "748bf3f5ecee61af20901338fa7133d4"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.312686 20060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:43.313822 20085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.314109 20060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:43.314376 20060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "b60b419870264b5997888887b3177cac"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "55d4bce34c92bb7afba29fdb524ad650"
server_key_iv: "748bf3f5ecee61af20901338fa7133d4"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.314483 20060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:43.336297 20060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:43.338466 20060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:43.338652 20060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:43.338901 20060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:43.339239 20060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:43.339298 20060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.339345 20060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:43.339373 20060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.346388 20060 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:46849
I20260807 08:37:43.346453 20256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:46849 every 8 connection(s)
I20260807 08:37:43.346777 20060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:43.355060 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 20060
I20260807 08:37:43.355219 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:43.355463 11068 external_mini_cluster.cc:1428] Setting key 7ffe96c966b89150d188b5f17860fc7a
I20260807 08:37:43.357595 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.347681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:56495 (local address 127.10.207.62:35169)
0807 08:37:43.347779 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.347782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.347925 (+ 143us) server_negotiation.cc:407] Connection header received
0807 08:37:43.348552 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.348554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.348565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.348622 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.349728 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.350238 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.351430 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.351657 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.351732 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.354255 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.354256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.354257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.354263 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.356029 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.356539 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.356540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.356541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.356615 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.356892 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.356893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.356894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.357024 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.357101 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.357386 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.357498 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:37:43.357803 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.347583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.2:56495)
0807 08:37:43.347759 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.347811 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.348444 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.348699 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.348703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.349373 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.349634 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.349639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.350342 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.350343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.351331 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.351333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.351760 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.351761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.351876 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.352520 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.352530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.354164 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.356115 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.356149 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.356151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.356433 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.356706 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.356707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.356707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.356821 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.357100 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.357109 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.357251 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.357363 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.357534 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260807 08:37:43.358325 20258 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35169
I20260807 08:37:43.358441 20258 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:43.358680 20258 heartbeater.cc:511] Master 127.10.207.62:35169 requested a full tablet report, sending...
I20260807 08:37:43.359160 19775 ts_manager.cc:194] Registered new tserver with Master: b60b419870264b5997888887b3177cac (127.10.207.2:46849)
I20260807 08:37:43.359552 19775 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:56495
WARNING: no policy specified for oryx/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab.
Entry for principal oryx/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab.
I20260807 08:37:43.385602 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35169
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab
--principal=oryx/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:43.481462 20268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:43.481726 20268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:43.481763 20268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:43.483016 20268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:43.483086 20268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:43.483132 20268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:43.484714 20268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab
--principal=oryx/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:35169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:43.485015 20268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:43.485275 20268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:43.490897 20274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.490899 20273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:43.491155 20268 server_base.cc:1034] running on GCE node
W20260807 08:37:43.491214 20276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:43.491497 20268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:43.491796 20268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:43.493017 20268 hybrid_clock.cc:648] HybridClock initialized: now 1786091863492990 us; error 37 us; skew 500 ppm
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:43.496013 20268 init.cc:377] Logged in from keytab as oryx/127.10.207.3@KRBTEST.COM (short username oryx)
I20260807 08:37:43.496487 20268 webserver.cc:460] Webserver started at http://127.10.207.3:38045/ using document root <none> and password file <none>
I20260807 08:37:43.496752 20268 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:43.496824 20268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:43.496981 20268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:43.498095 20268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "8e27357895b849c783d74ea5b4e8edaa"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "f937b46e61ae5b784b008431f31f14c2"
server_key_iv: "847e1da18f4f5c2e79f071d7f5740bb2"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.498489 20268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "8e27357895b849c783d74ea5b4e8edaa"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "f937b46e61ae5b784b008431f31f14c2"
server_key_iv: "847e1da18f4f5c2e79f071d7f5740bb2"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.500123 20268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:43.501178 20284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.501464 20268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:43.501591 20268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "8e27357895b849c783d74ea5b4e8edaa"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "f937b46e61ae5b784b008431f31f14c2"
server_key_iv: "847e1da18f4f5c2e79f071d7f5740bb2"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.501677 20268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:43.517549 20268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:43.519634 20268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:43.519786 20268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:43.519975 20268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:43.520282 20268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:43.520323 20268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.520342 20268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:43.520355 20268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.526927 20268 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34065
I20260807 08:37:43.527070 20451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34065 every 8 connection(s)
I20260807 08:37:43.527325 20268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:43.532137 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 20268
I20260807 08:37:43.532271 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:43.532482 11068 external_mini_cluster.cc:1428] Setting key d31d9e444b847152612aae1bd9353ee8
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:43.537048 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.528138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:37219 (local address 127.10.207.62:35169)
0807 08:37:43.528247 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.528250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.528500 (+ 250us) server_negotiation.cc:407] Connection header received
0807 08:37:43.528964 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.528965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.528977 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.529038 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.529928 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.530520 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.531574 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.531775 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.531842 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.534270 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.534271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.534271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.534278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.535709 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.536138 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.536139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.536139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.536196 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.536471 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.536472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.536472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.536605 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.536674 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.536907 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.536976 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:37:43.537140 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.528124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.3:37219)
0807 08:37:43.528367 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.528404 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.528864 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.529089 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.529092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.529633 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.529805 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.529810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.530690 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.530691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.531464 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.531475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.531871 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.531872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.531987 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.532599 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.532609 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.534177 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.535804 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.535811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.535812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.536051 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.536271 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.536271 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.536271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.536370 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.536680 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.536685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.536787 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.536874 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.536989 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":151,"threads_started":1}
I20260807 08:37:43.537745 20452 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35169
I20260807 08:37:43.537882 20452 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:43.538120 20452 heartbeater.cc:511] Master 127.10.207.62:35169 requested a full tablet report, sending...
I20260807 08:37:43.538574 19775 ts_manager.cc:194] Registered new tserver with Master: 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3:34065)
I20260807 08:37:43.539053 19775 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:37219
I20260807 08:37:43.544512 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.207.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.61.keytab.
Entry for principal oryx/127.10.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.61.keytab.
I20260807 08:37:43.573285 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:43817
--webserver_interface=127.10.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.61.keytab
--principal=oryx/127.10.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:35169,127.10.207.61:43817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:43.664268 20458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:43.664491 20458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:43.664515 20458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:43.665632 20458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:43.665681 20458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:43.665699 20458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:43.665714 20458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:43.665728 20458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:43.667328 20458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:35169,127.10.207.61:43817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.61.keytab
--principal=oryx/127.10.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:43817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
W20260807 08:37:43.667441 20458 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.207.62:35169,127.10.207.61:43817'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 08:37:43.667608 20458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:43.667872 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:43.673204 20464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.673318 20467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.673203 20465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:43.673408 20458 server_base.cc:1034] running on GCE node
I20260807 08:37:43.673774 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:43.674033 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:43.675196 20458 hybrid_clock.cc:648] HybridClock initialized: now 1786091863675181 us; error 37 us; skew 500 ppm
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:43.677723 20458 init.cc:377] Logged in from keytab as oryx/127.10.207.61@KRBTEST.COM (short username oryx)
I20260807 08:37:43.678179 20458 webserver.cc:460] Webserver started at http://127.10.207.61:34703/ using document root <none> and password file <none>
I20260807 08:37:43.678422 20458 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:43.678494 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:43.678655 20458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:43.679701 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/instance:
uuid: "cdd89981a4ae4903bf1af7630e7c02e8"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "4df39ff2bc79c559bed720caadc66111"
server_key_iv: "2d254c83c01efe821372b3c2cfc6040f"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.680155 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal/instance:
uuid: "cdd89981a4ae4903bf1af7630e7c02e8"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "4df39ff2bc79c559bed720caadc66111"
server_key_iv: "2d254c83c01efe821372b3c2cfc6040f"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.681592 20458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:43.682520 20475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.682857 20458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:43.682972 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
uuid: "cdd89981a4ae4903bf1af7630e7c02e8"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "4df39ff2bc79c559bed720caadc66111"
server_key_iv: "2d254c83c01efe821372b3c2cfc6040f"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.683063 20458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:43.713243 20458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:43.715521 20458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:43.715664 20458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:43.721235 20458 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.61:43817
I20260807 08:37:43.721287 20552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.61:43817 every 8 connection(s)
I20260807 08:37:43.721693 20458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
I20260807 08:37:43.723064 20554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:43.724840 20554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:43.731094 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 20458
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.61@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.207.61@KRBTEST.COM
I20260807 08:37:43.740083 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.725705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.10.207.62:35169)
0807 08:37:43.725777 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.725780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.725953 (+ 173us) server_negotiation.cc:407] Connection header received
0807 08:37:43.726463 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.726464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.726479 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.726622 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.727826 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.728736 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.732374 (+ 3638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.732709 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.732831 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.735612 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.735614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.735615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.735623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.737772 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.738411 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.738412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.738413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.738481 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.738884 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.738885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.738885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.739131 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.739226 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.739816 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.739935 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:37:43.740173 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.725604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44144)
0807 08:37:43.725810 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.725856 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.726361 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.726830 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.726836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.727326 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.727667 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.727674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.728880 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.728882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.730445 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.730449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.732841 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.732843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.732989 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.733656 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.733665 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.735488 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.737896 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.737904 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.737906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.738302 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.738623 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.738624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.738624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.738786 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.739255 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.739263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.739457 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.739602 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.739882 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":124,"threads_started":1}
I20260807 08:37:43.741490 20554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:43.743602 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.731933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.10.207.61:43817)
0807 08:37:43.732267 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.732273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.732322 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:37:43.732354 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.732357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.732384 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.732484 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.733646 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.734833 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.736017 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.736510 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.736591 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.739609 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.739617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.739618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.739626 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.741648 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.742128 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.742132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.742132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.742205 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.742443 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.742444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.742445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.742712 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.742915 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.743261 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.743371 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":202,"threads_started":1}
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.61@KRBTEST.COM for oryx/127.10.207.61@KRBTEST.COM
I20260807 08:37:43.750677 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.741928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.10.207.61:43817)
0807 08:37:43.742146 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.742149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.742197 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:37:43.742224 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.742225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.742242 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.742359 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.742747 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.743398 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.744536 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.744750 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.744828 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.747355 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.747360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.747361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.747373 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.749457 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.749838 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.749839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.749839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.749895 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.750092 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.750093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.750094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.750204 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.750281 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.750431 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.750538 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":128,"threads_started":1}
I20260807 08:37:43.750767 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.741880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45866)
0807 08:37:43.742092 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.742103 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.742143 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.742378 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.742379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.742591 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.742659 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.742660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.743495 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.743497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.744418 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.744421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.744845 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.744847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.744946 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.745380 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.745385 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.747125 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.749569 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.749570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.749570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.749744 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.749963 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.749964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.749965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.750020 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.750272 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.750273 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.750314 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.750409 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.750634 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":128}
I20260807 08:37:43.752352 20554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Bootstrap starting.
I20260807 08:37:43.753126 20554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:43.753414 20554 log.cc:824] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:43.754184 20554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: No bootstrap required, opened a new log
I20260807 08:37:43.755694 20554 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } }
I20260807 08:37:43.755818 20554 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:43.755844 20554 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd89981a4ae4903bf1af7630e7c02e8, State: Initialized, Role: FOLLOWER
I20260807 08:37:43.755992 20554 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } }
I20260807 08:37:43.756343 20554 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:43.756356 20564 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } } }
I20260807 08:37:43.756532 20564 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:37:43.758178 20584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:43.758302 20584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:43.760298 20458 master_runner.cc:428] Detected that this master cdd89981a4ae4903bf1af7630e7c02e8 is joining an existing cluster
I20260807 08:37:43.760387 20458 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.207.61:43817
I20260807 08:37:43.767508 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.761393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.10.207.62:35169)
0807 08:37:43.761459 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.761462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.761695 (+ 233us) server_negotiation.cc:407] Connection header received
0807 08:37:43.761806 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.761807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.761821 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.761854 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.762249 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.762838 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.763577 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.763784 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.763877 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.764802 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.764803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.764803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.764810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.766232 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.766626 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.766627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.766627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.766693 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.766929 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.766930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.766930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.767061 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.767105 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.767336 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.767411 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:37:43.768023 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.761348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44148)
0807 08:37:43.761627 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.761639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.761724 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.761942 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.761943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.762104 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.762167 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.762168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.762927 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.762928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.763492 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.763494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.763881 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.763882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.763956 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.764376 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.764387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.764673 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.766342 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.766343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.766343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.766517 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.766770 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.766771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.766771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.766851 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.767157 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.767158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.767200 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.767790 (+ 590us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.767893 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":219,"threads_started":1}
I20260807 08:37:43.773273 20564 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:43.773412 20564 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } }
I20260807 08:37:43.773720 20564 leader_election.cc:288] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169)
I20260807 08:37:43.774235 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f518004fae84e6b951b3255e3e546a5" is_pre_election: true
I20260807 08:37:43.774786 20476 leader_election.cc:302] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: cdd89981a4ae4903bf1af7630e7c02e8; no voters: 9f518004fae84e6b951b3255e3e546a5
I20260807 08:37:43.775046 20564 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091862, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.62@KRBTEST.COM for oryx/127.10.207.61@KRBTEST.COM
I20260807 08:37:43.779263 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.761740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45876 (local address 127.10.207.61:43817)
0807 08:37:43.761773 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.761775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.766708 (+ 4933us) server_negotiation.cc:407] Connection header received
0807 08:37:43.766796 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.766797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.766808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.766844 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.767244 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.767964 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.769351 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.769691 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.769733 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.771268 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.771269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.771269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.771275 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.773039 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.773414 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.773415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.773416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.774046 (+ 630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.778745 (+ 4699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.778746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.778746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.778858 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.778895 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.779067 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.779162 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:37:43.779331 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.770791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.10.207.61:43817)
0807 08:37:43.770824 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.770826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.771050 (+ 224us) server_negotiation.cc:407] Connection header received
0807 08:37:43.771181 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.771182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.771191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.771221 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.771885 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.772658 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.773721 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.774050 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.774132 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.776183 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.776184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.776185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.776192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.777919 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.778305 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.778307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.778307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.778362 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.778647 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.778647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.778648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.778772 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.778813 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.779095 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.779212 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":17}
I20260807 08:37:43.779585 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.770788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45892)
0807 08:37:43.770939 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.770989 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.771086 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.771301 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.771304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.771716 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.771782 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.771785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.772760 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.772761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.773639 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.773642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.774146 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.774147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.774223 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.774721 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.774727 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.776087 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.778002 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.778009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.778012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.778220 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.778453 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.778454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.778454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.778520 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.778864 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.778872 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.778927 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.779125 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.779482 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260807 08:37:43.780000 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.761807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45876)
0807 08:37:43.766653 (+ 4846us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.766664 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.766732 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.766897 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.766898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.767096 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.767170 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.767171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.768068 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.768069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.769259 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.769262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.769809 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.769810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.769871 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.770715 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.770720 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.771190 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.773120 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.773120 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.773121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.773340 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.778604 (+ 5264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.778606 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.778606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.778671 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.778934 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.778935 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.778966 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.779794 (+ 828us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.779906 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4822,"thread_start_us":127,"threads_started":1}
I20260807 08:37:43.780222 19775 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.207.61:43817: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8"
member_type: NON_VOTER
last_known_addr {
host: "127.10.207.61"
port: 43817
}
attrs {
promote: true
}
}
dest_uuid: "9f518004fae84e6b951b3255e3e546a5"
cas_config_opid_index: -1
I20260807 08:37:43.780567 19775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } }
I20260807 08:37:43.782737 20595 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cdd89981a4ae4903bf1af7630e7c02e8 (127.10.207.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } }
I20260807 08:37:43.782991 20511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:43.783874 20511 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Refusing update from remote peer 9f518004fae84e6b951b3255e3e546a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 08:37:43.783917 20595 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.207.61:43817
I20260807 08:37:43.784196 20458 master.cc:537] Master@127.10.207.61:43817 shutting down...
I20260807 08:37:43.785292 20595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } }
I20260807 08:37:43.786342 20595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:43.786567 20595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:43.787174 20458 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:43.787259 20458 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:43.787309 20458 tablet_replica.cc:330] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: stopping tablet replica
W20260807 08:37:43.787740 19755 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 08:37:43.790241 20458 master.cc:552] Master@127.10.207.61:43817 shutdown complete.
I20260807 08:37:43.790320 20458 master_runner.cc:305] Clearing existing system tablet
I20260807 08:37:43.790510 20458 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:43.790678 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:43.791368 20458 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:43.791899 20603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.792068 20458 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:43.792132 20458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:43.792243 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
uuid: "cdd89981a4ae4903bf1af7630e7c02e8"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "4df39ff2bc79c559bed720caadc66111"
server_key_iv: "2d254c83c01efe821372b3c2cfc6040f"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.793015 20458 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:43.794551 20458 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:37:43.794857 20458 log.cc:1196] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 08:37:43.795202 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.794857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45910)
0807 08:37:43.795004 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.795075 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 08:37:43.795374 19755 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 -> Peer cdd89981a4ae4903bf1af7630e7c02e8 (127.10.207.61:43817): Couldn't send request to peer cdd89981a4ae4903bf1af7630e7c02e8. Status: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:37:43.795607 20458 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Deleting consensus metadata
I20260807 08:37:43.795784 20458 master_runner.cc:315] Copying system tablet from 127.10.207.62:35169
I20260807 08:37:43.796473 20458 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: tablet copy: Beginning tablet copy session from remote peer at address 127.10.207.62:35169
I20260807 08:37:43.803349 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.796828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.10.207.62:35169)
0807 08:37:43.796877 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.796879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.797017 (+ 138us) server_negotiation.cc:407] Connection header received
0807 08:37:43.797100 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.797101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.797113 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.797175 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.797505 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.798254 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.799233 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.799442 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.799515 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.800499 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.800501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.800501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.800508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.801943 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.802312 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.802313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.802314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.802368 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.802606 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.802607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.802608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.802722 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.802798 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.803171 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.803258 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:37:43.803369 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.796776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44152)
0807 08:37:43.796935 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:43.796945 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:43.797034 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:43.797183 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:43.797183 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:43.797368 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:43.797419 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.797420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.798392 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.798393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.799108 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:43.799111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.799537 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.799538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:43.799605 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.800083 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:43.800088 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:43.800384 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:43.802058 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.802059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.802060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.802234 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.802445 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:43.802446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:43.802447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:43.802504 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:43.802790 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:43.802791 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:43.802835 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 08:37:43.803132 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 08:37:43.803260 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260807 08:37:43.804073 19803 tablet_copy_service.cc:140] P 9f518004fae84e6b951b3255e3e546a5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cdd89981a4ae4903bf1af7630e7c02e8 ({username='oryx', principal='oryx/127.10.207.61@KRBTEST.COM'} at 127.0.0.1:44152)
I20260807 08:37:43.804195 19803 tablet_copy_service.cc:161] P 9f518004fae84e6b951b3255e3e546a5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cdd89981a4ae4903bf1af7630e7c02e8 at {username='oryx', principal='oryx/127.10.207.61@KRBTEST.COM'} at 127.0.0.1:44152: session id = cdd89981a4ae4903bf1af7630e7c02e8-00000000000000000000000000000000
I20260807 08:37:43.805023 19803 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:37:43.805733 20458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:37:43.807058 20458 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: tablet copy: Starting download of 0 data blocks...
I20260807 08:37:43.807232 20458 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: tablet copy: Starting download of 1 WAL segments...
I20260807 08:37:43.808425 20458 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:37:43.809147 19803 tablet_copy_service.cc:342] P 9f518004fae84e6b951b3255e3e546a5: Request end of tablet copy session cdd89981a4ae4903bf1af7630e7c02e8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.207.61@KRBTEST.COM'} at 127.0.0.1:44152
I20260807 08:37:43.809221 19803 tablet_copy_service.cc:433] P 9f518004fae84e6b951b3255e3e546a5: ending tablet copy session cdd89981a4ae4903bf1af7630e7c02e8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cdd89981a4ae4903bf1af7630e7c02e8
I20260807 08:37:43.811794 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:43.812857 20612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.815480 20613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:43.815689 20615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:43.815860 20458 server_base.cc:1034] running on GCE node
I20260807 08:37:43.815949 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:43.816088 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:43.817194 20458 hybrid_clock.cc:648] HybridClock initialized: now 1786091863817184 us; error 22 us; skew 500 ppm
Aug 07 08:37:43 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091863, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:43.818683 20458 init.cc:377] Logged in from keytab as oryx/127.10.207.61@KRBTEST.COM (short username oryx)
I20260807 08:37:43.819003 20458 webserver.cc:460] Webserver started at http://127.10.207.61:45607/ using document root <none> and password file <none>
I20260807 08:37:43.819125 20458 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:43.819182 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:43.819756 20458 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.820339 20624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:43.820467 20458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:37:43.820514 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
uuid: "cdd89981a4ae4903bf1af7630e7c02e8"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "4df39ff2bc79c559bed720caadc66111"
server_key_iv: "2d254c83c01efe821372b3c2cfc6040f"
server_key_version: "encryptionkey@0"
I20260807 08:37:43.820570 20458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:43.844544 20458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:43.846414 20458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:43.849913 20458 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.61:43817
I20260807 08:37:43.849961 20698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.61:43817 every 8 connection(s)
I20260807 08:37:43.850268 20458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
I20260807 08:37:43.851480 20699 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:37:43.852066 20699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Bootstrap starting.
I20260807 08:37:43.855815 20699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:43.856118 20699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: Bootstrap complete.
I20260807 08:37:43.856346 20699 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } }
I20260807 08:37:43.856447 20699 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cdd89981a4ae4903bf1af7630e7c02e8, State: Initialized, Role: LEARNER
I20260807 08:37:43.856487 20699 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } }
I20260807 08:37:43.856698 20699 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:43.856956 20703 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } }
I20260807 08:37:43.857084 20703 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:37:43.858842 20719 catalog_manager.cc:1259] Loaded cluster ID: 267afb8bff0b4f77ad061cf56a20e1ec
I20260807 08:37:43.858928 20719 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: loading cluster ID for follower catalog manager: success
I20260807 08:37:43.859989 20719 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: acquiring CA information for follower catalog manager: success
I20260807 08:37:43.860352 20719 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:37:43.904057 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:43.896274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.10.207.61:43817)
0807 08:37:43.896445 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:43.896447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:43.896454 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:37:43.896484 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:43.896485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:43.896495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:43.896584 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:43.897308 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.898033 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.899086 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:43.899407 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:43.899492 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:43.901117 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:43.901119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:43.901119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:43.901138 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:43.902773 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.903135 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.903136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.903136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.903197 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:43.903410 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:43.903411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:43.903417 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:43.903546 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:43.903662 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:43.903770 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 08:37:43.903865 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for oryx/127.10.207.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.60.keytab.
Entry for principal oryx/127.10.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.60.keytab.
I20260807 08:37:43.938269 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:46355
--webserver_interface=127.10.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.60.keytab
--principal=oryx/127.10.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:44.024307 20725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:44.024497 20725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:44.024518 20725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:44.025557 20725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:44.025597 20725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:44.025612 20725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:44.025625 20725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:44.025637 20725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:44.027132 20725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.60.keytab
--principal=oryx/127.10.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:46355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:44.027321 20725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:44.027478 20725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:44.032768 20734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.032779 20738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:44.033082 20725 server_base.cc:1034] running on GCE node
W20260807 08:37:44.032779 20735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:44.033516 20725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:44.033840 20725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:44.035043 20725 hybrid_clock.cc:648] HybridClock initialized: now 1786091864035024 us; error 34 us; skew 500 ppm
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:44.037882 20725 init.cc:377] Logged in from keytab as oryx/127.10.207.60@KRBTEST.COM (short username oryx)
I20260807 08:37:44.038511 20725 webserver.cc:460] Webserver started at http://127.10.207.60:36877/ using document root <none> and password file <none>
I20260807 08:37:44.038851 20725 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:44.038926 20725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:44.039096 20725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:44.040431 20725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/instance:
uuid: "36b08ff551244486b2da6e53cbf0ac1d"
format_stamp: "Formatted at 2026-08-07 08:37:44 on dist-test-slave-h6n0"
server_key: "d839d5017b347b6174db2aaf8be89217"
server_key_iv: "12e7de926ca2cffb1c9729e487873ba8"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.040903 20725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal/instance:
uuid: "36b08ff551244486b2da6e53cbf0ac1d"
format_stamp: "Formatted at 2026-08-07 08:37:44 on dist-test-slave-h6n0"
server_key: "d839d5017b347b6174db2aaf8be89217"
server_key_iv: "12e7de926ca2cffb1c9729e487873ba8"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.042475 20725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 08:37:44.043609 20753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.043896 20725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:44.044029 20725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
uuid: "36b08ff551244486b2da6e53cbf0ac1d"
format_stamp: "Formatted at 2026-08-07 08:37:44 on dist-test-slave-h6n0"
server_key: "d839d5017b347b6174db2aaf8be89217"
server_key_iv: "12e7de926ca2cffb1c9729e487873ba8"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.044152 20725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:44.065335 20725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:44.067863 20725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:44.068066 20725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:44.074994 20725 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.60:46355
I20260807 08:37:44.075080 20823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.60:46355 every 8 connection(s)
I20260807 08:37:44.075507 20725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
I20260807 08:37:44.076678 20825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:44.078297 20825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:44.084980 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 20725
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.60@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:44.090893 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.079181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.10.207.62:35169)
0807 08:37:44.079301 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.079313 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.079668 (+ 355us) server_negotiation.cc:407] Connection header received
0807 08:37:44.080454 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.080455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.080472 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.080508 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.082919 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.083436 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.084415 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.084607 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.084688 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.087592 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.087593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.087594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.087601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.089165 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.089755 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.089756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.089757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.089816 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.090091 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.090092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.090093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.090255 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.090340 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.090681 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.090765 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:37:44.091084 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.079080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44158)
0807 08:37:44.079500 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.079548 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.080302 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.080718 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.080723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.081139 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.081461 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.081467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.083545 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.083546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.084316 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.084318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.084722 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.084723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.084909 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.085750 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.085772 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.087495 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.089259 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.089264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.089266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.089669 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.089896 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.089897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.089897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.090009 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.090340 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.090345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.090475 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.090649 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.090797 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":258,"threads_started":1}
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.207.60@KRBTEST.COM
I20260807 08:37:44.091701 20825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:37:44.095652 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.085593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.10.207.60:46355)
0807 08:37:44.085902 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.085905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.085921 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:37:44.085938 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.085940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.085957 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.086053 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.086718 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.087630 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.088698 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.089023 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.089097 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.092192 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.092199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.092200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.092214 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.094140 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.094596 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.094600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.094600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.094669 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.094908 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.094909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.094909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.095114 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.095290 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.095405 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.095537 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":224,"threads_started":1}
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.60@KRBTEST.COM for oryx/127.10.207.61@KRBTEST.COM
I20260807 08:37:44.099359 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.092000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45942)
0807 08:37:44.092057 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.092068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.092117 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.092288 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.092289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.092450 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.092515 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.092517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.093381 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.093382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.094270 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.094272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.094797 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.094798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.094860 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.095335 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.095340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.096765 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.098371 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.098372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.098372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.098614 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.098823 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.098823 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.098824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.098893 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.099116 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.099117 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.099155 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.099221 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.099299 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:37:44.099448 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.092101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.10.207.61:43817)
0807 08:37:44.092150 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.092152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.092161 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:44.092191 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.092192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.092204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.092262 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.092614 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.093284 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.094370 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.094690 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.094784 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.096873 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.096875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.096875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.096882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.098282 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.098686 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.098686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.098687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.098745 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.098944 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.098944 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.098944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.099034 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.099088 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.099246 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.099310 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:37:44.100003 20825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.60@KRBTEST.COM for oryx/127.10.207.60@KRBTEST.COM
I20260807 08:37:44.106712 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.100253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43018)
0807 08:37:44.100310 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.100321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.100360 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.100471 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.100472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.100611 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.100637 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.100638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.101271 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.101272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.101736 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.101738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.102110 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.102111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.102160 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.102578 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.102583 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.103726 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.105362 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.105363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.105363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.105666 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.105897 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.105898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.105898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.106025 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.106340 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.106340 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.106381 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.106442 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.106614 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 08:37:44.106853 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.100332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.10.207.60:46355)
0807 08:37:44.100376 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.100378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.100386 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:44.100405 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.100406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.100415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.100468 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.100719 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.101187 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.101819 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.102022 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.102106 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.103840 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.103841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.103842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.103848 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.105274 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.105752 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.105752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.105753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.105806 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.106132 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.106133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.106133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.106259 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.106331 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.106483 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.106712 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":26}
I20260807 08:37:44.108035 20825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Bootstrap starting.
I20260807 08:37:44.108958 20825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:44.109241 20825 log.cc:824] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:44.110198 20825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: No bootstrap required, opened a new log
I20260807 08:37:44.111677 20825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } }
I20260807 08:37:44.111793 20825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:44.111816 20825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b08ff551244486b2da6e53cbf0ac1d, State: Initialized, Role: FOLLOWER
I20260807 08:37:44.111956 20825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } }
I20260807 08:37:44.112269 20825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:44.112273 20830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } } }
I20260807 08:37:44.112483 20830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:37:44.114898 20845 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:37:44.114970 20845 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:37:44.117702 20725 master_runner.cc:428] Detected that this master 36b08ff551244486b2da6e53cbf0ac1d is joining an existing cluster
I20260807 08:37:44.117758 20725 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.207.60:46355
I20260807 08:37:44.147320 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 19868
I20260807 08:37:44.161228 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.119743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.10.207.62:35169)
0807 08:37:44.119804 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.119806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.119958 (+ 152us) server_negotiation.cc:407] Connection header received
0807 08:37:44.120068 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.120069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.120084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.120144 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.120621 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.121465 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.122614 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.122905 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.122944 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.124424 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.124425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.124426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.124434 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.126478 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.160060 (+ 33582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.160063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.160063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.160162 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.160477 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.160478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.160478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.160648 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.160731 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.161011 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.161122 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:37:44.161368 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.119710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44162)
0807 08:37:44.119870 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.119881 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.119953 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.120210 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.120211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.120406 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.120500 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.120501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.121597 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.121597 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.122482 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.122484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.123031 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.123032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.123108 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.123961 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.123965 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.124325 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.126601 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.126602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.126603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.159909 (+ 33306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.160264 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.160265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.160266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.160381 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.160743 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.160744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.160828 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.160969 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.161165 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260807 08:37:44.167933 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.119812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.10.207.61:43817)
0807 08:37:44.120438 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.120440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.120450 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:44.120472 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.120473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.120482 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.120554 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.120969 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.121689 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.122900 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.123190 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.123735 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.164308 (+ 40573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.164310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.164310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.164318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.166127 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.166577 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.166578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.166578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.166634 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.166992 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.166995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.166996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.167248 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.167316 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.167634 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.167770 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602}
I20260807 08:37:44.168507 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.119880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45958)
0807 08:37:44.120109 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.120119 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.120169 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.120593 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.120594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.120781 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.120851 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.120852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.121798 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.121799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.122802 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.122805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.123591 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.123592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.123667 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.163742 (+ 40075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.163748 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.164187 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.166230 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.166231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.166232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.166448 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.166734 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.166735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.166736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.166847 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.167392 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.167393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.167468 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.168138 (+ 670us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.168316 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":154,"threads_started":1}
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091862, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.62@KRBTEST.COM for oryx/127.10.207.60@KRBTEST.COM
I20260807 08:37:44.177168 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.165149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43030)
0807 08:37:44.165513 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.165527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.165580 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.166933 (+ 1353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.166935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.167103 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.167155 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.167158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.168419 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.168420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.169253 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.169256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.170177 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.170178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.170346 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.170941 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.170948 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.172792 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.175805 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.175806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.175806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.176043 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.176347 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.176348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.176348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.176420 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.176815 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.176816 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.176864 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.176948 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.177063 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337}
I20260807 08:37:44.177672 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.165720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.10.207.60:46355)
0807 08:37:44.166765 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.166767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.166775 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:44.166813 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.166813 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.166826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.167230 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.167272 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.168317 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.169689 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.170013 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.170086 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.173262 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.173265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.173265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.173274 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.175690 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.176148 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.176149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.176150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.176253 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.176507 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.176508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.176508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.176715 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.176821 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.177005 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.177132 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":766,"threads_started":1}
I20260807 08:37:44.178575 19775 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.207.60:46355: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d"
member_type: NON_VOTER
last_known_addr {
host: "127.10.207.60"
port: 46355
}
attrs {
promote: true
}
}
dest_uuid: "9f518004fae84e6b951b3255e3e546a5"
cas_config_opid_index: 5
I20260807 08:37:44.178973 19775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } }
I20260807 08:37:44.179497 20597 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.180258 20787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:44.180783 20597 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.207.60:46355
I20260807 08:37:44.180862 20597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.180920 20597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:44.181191 20725 master.cc:537] Master@127.10.207.60:46355 shutting down...
I20260807 08:37:44.181393 20787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Refusing update from remote peer 9f518004fae84e6b951b3255e3e546a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 08:37:44.181706 20596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:37:44.182408 19755 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 -> Peer 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60:46355): Couldn't send request to peer 36b08ff551244486b2da6e53cbf0ac1d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:37:44.183724 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.120250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.10.207.60:46355)
0807 08:37:44.120774 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.120776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.121087 (+ 311us) server_negotiation.cc:407] Connection header received
0807 08:37:44.121138 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.121139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.121148 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.121184 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.121596 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.122293 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.123283 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.123511 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.123576 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.125629 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.125630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.125631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.125637 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.127212 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.182863 (+ 55651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.182864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.182865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.182925 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.183151 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.183152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.183152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.183267 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.183340 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.183533 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.183633 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505}
I20260807 08:37:44.183874 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.120605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43026)
0807 08:37:44.120985 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.120994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.121040 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.121215 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.121216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.121426 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.121497 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.121499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.122375 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.122376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.123196 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.123198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.124447 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.124448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.124519 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.125086 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.125089 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.125523 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.127314 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.127315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.127316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.182757 (+ 55441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.183014 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.183014 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.183015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.183083 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.183342 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.183343 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.183385 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.183498 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.183713 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":85,"thread_start_us":120,"threads_started":1}
I20260807 08:37:44.187477 20725 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:44.187603 20725 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:44.187673 20725 tablet_replica.cc:330] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: stopping tablet replica
I20260807 08:37:44.188532 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:42489
--local_ip_for_outbound_sockets=127.10.207.1
--tserver_master_addrs=127.10.207.62:35169
--webserver_port=35023
--webserver_interface=127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab
--principal=oryx/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:44.192032 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.182737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.10.207.61:43817)
0807 08:37:44.182795 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.182798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.186756 (+ 3958us) server_negotiation.cc:407] Connection header received
0807 08:37:44.187304 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.187306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.187336 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.187384 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:44.188568 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.189311 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.190855 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.191510 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.191559 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.191646 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.191894 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.191949 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:37:44.192472 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.179973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45974)
0807 08:37:44.186688 (+ 6715us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.186702 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.187127 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.188388 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.188389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.188391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:44.188447 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.188449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.189420 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.189422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.190747 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.190750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.191712 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.191713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.191784 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.191808 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.192199 (+ 391us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.192259 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6692}
I20260807 08:37:44.192746 20661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 LEARNER]: Refusing update from remote peer 9f518004fae84e6b951b3255e3e546a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 08:37:44.193008 20596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 08:37:44.193725 20661 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.194182 20725 master.cc:552] Master@127.10.207.60:46355 shutdown complete.
I20260807 08:37:44.194242 20725 master_runner.cc:305] Clearing existing system tablet
I20260807 08:37:44.194378 20725 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:44.194435 20725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:44.195120 20867 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.195271 20867 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:37:44.195406 20703 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f518004fae84e6b951b3255e3e546a5. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: NON_VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: true } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.195468 20725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:37:44.195515 20703 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:37:44.195739 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.195572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43036)
0807 08:37:44.195659 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.195701 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:46355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:37:44.195976 20868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.196135 20725 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:44.196215 20725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.196239 20725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
uuid: "36b08ff551244486b2da6e53cbf0ac1d"
format_stamp: "Formatted at 2026-08-07 08:37:44 on dist-test-slave-h6n0"
server_key: "d839d5017b347b6174db2aaf8be89217"
server_key_iv: "12e7de926ca2cffb1c9729e487873ba8"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.197047 20725 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:44.198657 20725 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:37:44.198745 20725 log.cc:1196] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 08:37:44.198979 20725 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Deleting consensus metadata
I20260807 08:37:44.199133 20725 master_runner.cc:315] Copying system tablet from 127.10.207.62:35169
I20260807 08:37:44.199847 20725 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: tablet copy: Beginning tablet copy session from remote peer at address 127.10.207.62:35169
I20260807 08:37:44.204473 20596 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: attempting to promote NON_VOTER cdd89981a4ae4903bf1af7630e7c02e8 to VOTER
I20260807 08:37:44.204743 20596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } }
I20260807 08:37:44.205468 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.205304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43052)
0807 08:37:44.205401 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.205434 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:46355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 08:37:44.205969 20661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 LEARNER]: Refusing update from remote peer 9f518004fae84e6b951b3255e3e546a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 08:37:44.206351 20595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 08:37:44.206698 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.206578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43068)
0807 08:37:44.206649 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.206673 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:46355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 08:37:44.206995 19755 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 -> Peer 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60:46355): Couldn't send request to peer 36b08ff551244486b2da6e53cbf0ac1d. Status: Network error: Client connection negotiation failed: client connection to 127.10.207.60:46355: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:37:44.207310 20595 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cdd89981a4ae4903bf1af7630e7c02e8 (127.10.207.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.208340 20595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.208437 20595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:44.208508 20595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.208583 20595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:44.208899 20661 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cdd89981a4ae4903bf1af7630e7c02e8 (127.10.207.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.209240 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.200267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.10.207.62:35169)
0807 08:37:44.200313 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.200315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.200440 (+ 125us) server_negotiation.cc:407] Connection header received
0807 08:37:44.200522 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.200524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.200547 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.200615 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.201149 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.201896 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.203162 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.203463 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.203557 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.205236 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.205239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.205239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.205250 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.207746 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.208160 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.208161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.208162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.208220 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.208409 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.208410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.208410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.208729 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.208779 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.208992 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.209139 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:37:44.209753 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.200182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44166)
0807 08:37:44.200341 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.200355 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.200436 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.200675 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.200677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.200939 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.201037 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.201040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.202007 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.202008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.203055 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.203058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.203572 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.203573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.203671 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.204343 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.204351 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.205098 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.207842 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.207843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.207844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.208075 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.208288 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.208289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.208289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.208343 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.208828 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.208829 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.208877 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.209509 (+ 632us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.209646 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260807 08:37:44.209954 20703 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.210043 20703 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:44.210204 19803 tablet_copy_service.cc:140] P 9f518004fae84e6b951b3255e3e546a5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 36b08ff551244486b2da6e53cbf0ac1d ({username='oryx', principal='oryx/127.10.207.60@KRBTEST.COM'} at 127.0.0.1:44166)
I20260807 08:37:44.210273 19803 tablet_copy_service.cc:161] P 9f518004fae84e6b951b3255e3e546a5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 36b08ff551244486b2da6e53cbf0ac1d at {username='oryx', principal='oryx/127.10.207.60@KRBTEST.COM'} at 127.0.0.1:44166: session id = 36b08ff551244486b2da6e53cbf0ac1d-00000000000000000000000000000000
I20260807 08:37:44.210979 19803 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:37:44.211558 20725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:37:44.212815 20725 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: tablet copy: Starting download of 0 data blocks...
I20260807 08:37:44.212973 20725 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: tablet copy: Starting download of 1 WAL segments...
I20260807 08:37:44.213935 20725 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:37:44.214849 19803 tablet_copy_service.cc:342] P 9f518004fae84e6b951b3255e3e546a5: Request end of tablet copy session 36b08ff551244486b2da6e53cbf0ac1d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.207.60@KRBTEST.COM'} at 127.0.0.1:44166
I20260807 08:37:44.214915 19803 tablet_copy_service.cc:433] P 9f518004fae84e6b951b3255e3e546a5: ending tablet copy session 36b08ff551244486b2da6e53cbf0ac1d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 36b08ff551244486b2da6e53cbf0ac1d
I20260807 08:37:44.218695 20725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:44.219908 20876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.220005 20877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.220216 20881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:44.220794 20725 server_base.cc:1034] running on GCE node
I20260807 08:37:44.220909 20725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:44.221066 20725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:44.222194 20725 hybrid_clock.cc:648] HybridClock initialized: now 1786091864222185 us; error 27 us; skew 500 ppm
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:44.223845 20725 init.cc:377] Logged in from keytab as oryx/127.10.207.60@KRBTEST.COM (short username oryx)
I20260807 08:37:44.224205 20725 webserver.cc:460] Webserver started at http://127.10.207.60:44787/ using document root <none> and password file <none>
I20260807 08:37:44.224340 20725 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:44.224377 20725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:44.225190 20725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:44.225873 20894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.226089 20725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:37:44.226147 20725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
uuid: "36b08ff551244486b2da6e53cbf0ac1d"
format_stamp: "Formatted at 2026-08-07 08:37:44 on dist-test-slave-h6n0"
server_key: "d839d5017b347b6174db2aaf8be89217"
server_key_iv: "12e7de926ca2cffb1c9729e487873ba8"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.226212 20725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:44.242048 20725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:44.244095 20725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:44.248445 20725 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.60:46355
I20260807 08:37:44.248554 20962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.60:46355 every 8 connection(s)
I20260807 08:37:44.248857 20725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
I20260807 08:37:44.250237 20963 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:37:44.250785 20963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Bootstrap starting.
I20260807 08:37:44.253962 20963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:44.254189 20963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Bootstrap complete.
I20260807 08:37:44.254387 20963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } }
I20260807 08:37:44.254467 20963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 36b08ff551244486b2da6e53cbf0ac1d, State: Initialized, Role: LEARNER
I20260807 08:37:44.254503 20963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } }
I20260807 08:37:44.254727 20963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:44.259210 20986 catalog_manager.cc:1259] Loaded cluster ID: 267afb8bff0b4f77ad061cf56a20e1ec
I20260807 08:37:44.259282 20986 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: loading cluster ID for follower catalog manager: success
I20260807 08:37:44.260407 20986 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: acquiring CA information for follower catalog manager: success
I20260807 08:37:44.260751 20986 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:37:44.262660 20968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.262794 20968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: This master's current role is: LEARNER
W20260807 08:37:44.309538 20863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:44.309774 20863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:44.309798 20863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:44.311079 20863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:44.311129 20863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:44.311158 20863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:44.312712 20863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab
--principal=oryx/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:42489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=35023
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:44.312984 20863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:44.313270 20863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:44.319005 20992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.319260 20993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.319082 20995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:44.319547 20863 server_base.cc:1034] running on GCE node
I20260807 08:37:44.319849 20863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:44.320188 20863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:44.321420 20863 hybrid_clock.cc:648] HybridClock initialized: now 1786091864321377 us; error 50 us; skew 500 ppm
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:44.324307 20863 init.cc:377] Logged in from keytab as oryx/127.10.207.1@KRBTEST.COM (short username oryx)
I20260807 08:37:44.324800 20863 webserver.cc:460] Webserver started at http://127.10.207.1:35023/ using document root <none> and password file <none>
I20260807 08:37:44.325013 20863 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:44.325078 20863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:44.327060 20863 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:44.327924 21010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.328223 20863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:44.328341 20863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "9bb5534a84574f759c0644363266b675"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "9a3f5f4ba77a2eac29aad817182ade8a"
server_key_iv: "bd7d0eee5d16942e8605467ff41585b3"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.328764 20863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:44.348759 20863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:44.351141 20863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:44.351290 20863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:44.351560 20863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:44.351898 20863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:44.352043 20863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.352259 20863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:44.352332 20863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.360702 20258 heartbeater.cc:503] Master 127.10.207.62:35169 was elected leader, sending a full tablet report...
I20260807 08:37:44.362157 20863 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:42489
I20260807 08:37:44.362231 21173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:42489 every 8 connection(s)
I20260807 08:37:44.362610 20863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:44.370231 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 20863
I20260807 08:37:44.370433 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 20060
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.61@KRBTEST.COM
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.60@KRBTEST.COM
I20260807 08:37:44.378855 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.363419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:39973 (local address 127.10.207.62:35169)
0807 08:37:44.363653 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.363656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.363917 (+ 261us) server_negotiation.cc:407] Connection header received
0807 08:37:44.365232 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.365233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.365249 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.365307 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.366346 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.366957 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368406 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.368607 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368674 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.373253 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.373254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.373255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.373262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.377165 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.377698 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.377699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.377700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.377756 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.378045 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.378046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.378046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.378183 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.378227 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.378676 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.378755 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260807 08:37:44.379050 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.363431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.10.207.1:42929)
0807 08:37:44.363760 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.363876 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.365101 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.365483 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.365484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.365941 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.366212 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.366218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.367481 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.367482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.368185 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.368187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.368696 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.368850 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.369618 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.369622 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.375560 (+ 5938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.377597 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.377597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.377598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.377827 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.378034 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.378035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.378035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.378090 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.378382 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.378383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.378446 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.378625 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.378802 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":22,"thread_start_us":198,"threads_started":1}
I20260807 08:37:44.379179 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.363429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.1:39973)
0807 08:37:44.363768 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.363819 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.365087 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.365411 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.365416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.365951 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.366214 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.366222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.367081 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.367082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.368285 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.368289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.368760 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.368850 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.369605 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.369617 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.372729 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.377284 (+ 4555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.377292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.377294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.377612 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.377830 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.377831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.377831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.377958 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.378274 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.378281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.378446 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.378976 (+ 530us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.379073 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":11,"thread_start_us":248,"threads_started":1}
I20260807 08:37:44.379391 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.363487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:42929 (local address 127.10.207.60:46355)
0807 08:37:44.364167 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.364169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.364180 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:44.365379 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.365380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.365393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.365472 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.366314 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.367400 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368278 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.368583 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368657 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.375682 (+ 7025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.375684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.375684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.375693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.377504 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.377904 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.377905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.377905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.377965 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.378157 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.378157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.378158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.378301 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.378359 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.379181 (+ 822us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.379277 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":90,"threads_started":1}
I20260807 08:37:44.379630 21175 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35169
I20260807 08:37:44.379717 21175 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.379856 21175 heartbeater.cc:511] Master 127.10.207.62:35169 requested a full tablet report, sending...
I20260807 08:37:44.380405 19775 ts_manager.cc:194] Re-registered known tserver with Master: 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489)
I20260807 08:37:44.380765 19775 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:39973
I20260807 08:37:44.388800 21174 heartbeater.cc:347] Connected to a master server at 127.10.207.60:46355
I20260807 08:37:44.388865 21174 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.388945 21174 heartbeater.cc:511] Master 127.10.207.60:46355 requested a full tablet report, sending...
I20260807 08:37:44.390327 20919 ts_manager.cc:194] Registered new tserver with Master: 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489)
I20260807 08:37:44.390827 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.364090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:48861 (local address 127.10.207.61:43817)
0807 08:37:44.364167 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.364169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.364180 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:44.365243 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.365245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.365261 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.365306 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.366370 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.367175 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368372 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.368664 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368726 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.373241 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.373243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.373244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.373253 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.389142 (+ 15889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.389647 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.389650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.389650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.389747 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.390012 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.390013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.390013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.390221 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.390307 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.390608 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.390707 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:37:44.390841 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.363869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.10.207.1:48861)
0807 08:37:44.364052 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.364066 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.365103 (+ 1037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.365411 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.365416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.365935 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.366214 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.366222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.367276 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.367276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.368285 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.368294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.368787 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.368788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.368860 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.369685 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.369690 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.373143 (+ 3453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.389255 (+ 16112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.389256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.389257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.389500 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.389846 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.389847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.389848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.389926 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.390328 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.390329 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.390379 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.390583 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.390709 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":19,"thread_start_us":96,"threads_started":1}
I20260807 08:37:44.391666 21176 heartbeater.cc:347] Connected to a master server at 127.10.207.61:43817
I20260807 08:37:44.391743 21176 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.391814 21176 heartbeater.cc:511] Master 127.10.207.61:43817 requested a full tablet report, sending...
I20260807 08:37:44.392253 20647 ts_manager.cc:194] Registered new tserver with Master: 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489)
I20260807 08:37:44.395546 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:46849
--local_ip_for_outbound_sockets=127.10.207.2
--tserver_master_addrs=127.10.207.62:35169
--webserver_port=38479
--webserver_interface=127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab
--principal=oryx/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:44.490252 21187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:44.490481 21187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:44.490505 21187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:44.491715 21187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:44.491766 21187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:44.491794 21187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:44.493302 21187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab
--principal=oryx/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:46849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=38479
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:44.493516 21187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:44.493793 21187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:44.499414 21195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.499456 21194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.499414 21199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:44.499953 21187 server_base.cc:1034] running on GCE node
I20260807 08:37:44.500203 21187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:44.500486 21187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:44.501667 21187 hybrid_clock.cc:648] HybridClock initialized: now 1786091864501636 us; error 78 us; skew 500 ppm
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:44.504914 21187 init.cc:377] Logged in from keytab as oryx/127.10.207.2@KRBTEST.COM (short username oryx)
I20260807 08:37:44.505471 21187 webserver.cc:460] Webserver started at http://127.10.207.2:38479/ using document root <none> and password file <none>
I20260807 08:37:44.505766 21187 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:44.505864 21187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:44.507565 21187 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:44.508365 21213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.508589 21187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:44.508726 21187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "b60b419870264b5997888887b3177cac"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "55d4bce34c92bb7afba29fdb524ad650"
server_key_iv: "748bf3f5ecee61af20901338fa7133d4"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.509163 21187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:44.520680 21187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:44.523458 21187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:44.523669 21187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:44.523949 21187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:44.524343 21187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:44.524402 21187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.524444 21187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:44.524492 21187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.533031 21187 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:46849
I20260807 08:37:44.533090 21372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:46849 every 8 connection(s)
I20260807 08:37:44.533486 21187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:44.539896 20452 heartbeater.cc:503] Master 127.10.207.62:35169 was elected leader, sending a full tablet report...
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.61@KRBTEST.COM
I20260807 08:37:44.542126 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 21187
I20260807 08:37:44.542300 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 20268
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.60@KRBTEST.COM
I20260807 08:37:44.546581 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.534297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:50763 (local address 127.10.207.62:35169)
0807 08:37:44.534390 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.534393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.534660 (+ 267us) server_negotiation.cc:407] Connection header received
0807 08:37:44.535491 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.535492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.535507 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.535573 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.536689 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.537542 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.538608 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.538904 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.538976 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.542017 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.542019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.542020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.542028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.544280 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.544991 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.544993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.544993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.545072 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.545493 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.545495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.545495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.545681 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.545743 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.546269 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.546398 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:37:44.547372 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.534272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.2:50763)
0807 08:37:44.534468 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.534515 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.535358 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.535657 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.535661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.536311 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.536531 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.536537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.537646 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.537647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.538468 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.538472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.539010 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.539011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.539190 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.539983 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.539993 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.541884 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.544429 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.544440 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.544444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.544879 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.545174 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.545175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.545175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.545381 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.545812 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.545820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.546048 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.546719 (+ 671us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.546981 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":33,"thread_start_us":101,"threads_started":1}
I20260807 08:37:44.551009 21375 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35169
I20260807 08:37:44.551167 21375 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.551409 21375 heartbeater.cc:511] Master 127.10.207.62:35169 requested a full tablet report, sending...
I20260807 08:37:44.551783 19775 ts_manager.cc:194] Re-registered known tserver with Master: b60b419870264b5997888887b3177cac (127.10.207.2:46849)
I20260807 08:37:44.552304 19775 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:50763
I20260807 08:37:44.557962 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.534942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:56943 (local address 127.10.207.60:46355)
0807 08:37:44.534988 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.534990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.535008 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:37:44.535765 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.535767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.535779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.535839 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.537663 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.538380 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.539321 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.539598 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.539681 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.554464 (+ 14783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.554465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.554466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.554475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.556534 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.556991 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.557002 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.557003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.557065 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.557283 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.557284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.557284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.557444 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.557490 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.557703 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.557826 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:37:44.558632 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.534697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.10.207.2:56943)
0807 08:37:44.534945 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.534957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.535360 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.535884 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.535885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.536235 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.536531 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.536537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.538484 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.538485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.539218 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.539221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.539697 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.539698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.539799 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.540445 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.540449 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.554315 (+ 13866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.556618 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.556619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.556620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.556900 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.557142 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.557143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.557143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.557211 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.557540 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.557541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.557586 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.558312 (+ 726us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.558457 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":9,"thread_start_us":98,"threads_started":1}
I20260807 08:37:44.559303 21374 heartbeater.cc:347] Connected to a master server at 127.10.207.60:46355
I20260807 08:37:44.559393 21374 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.559476 21374 heartbeater.cc:511] Master 127.10.207.60:46355 requested a full tablet report, sending...
I20260807 08:37:44.559751 20919 ts_manager.cc:194] Registered new tserver with Master: b60b419870264b5997888887b3177cac (127.10.207.2:46849)
I20260807 08:37:44.560211 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.534954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:33081 (local address 127.10.207.61:43817)
0807 08:37:44.535000 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.535003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.535011 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:44.535636 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.535637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.535645 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.535708 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.536709 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.537251 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.538735 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.539069 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.539101 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.556466 (+ 17365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.556469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.556470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.556483 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.558693 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.559116 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.559118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.559118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.559198 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.559510 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.559511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.559512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.559697 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.559761 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.560057 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.560146 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:37:44.560571 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.534273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.10.207.2:33081)
0807 08:37:44.534499 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.534519 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.535364 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.535746 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.535747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.536260 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.536531 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.536538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.537393 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.537394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.538637 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.538640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.539164 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.539165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.539223 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.540406 (+ 1183us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.540414 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.542336 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.558792 (+ 16456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.558793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.558793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.559006 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.559294 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.559296 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.559296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.559422 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.559804 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.559805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.559858 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.559947 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.560135 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":3,"thread_start_us":146,"threads_started":1}
I20260807 08:37:44.561017 21377 heartbeater.cc:347] Connected to a master server at 127.10.207.61:43817
I20260807 08:37:44.561103 21377 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.561153 21377 heartbeater.cc:511] Master 127.10.207.61:43817 requested a full tablet report, sending...
I20260807 08:37:44.561519 20647 ts_manager.cc:194] Registered new tserver with Master: b60b419870264b5997888887b3177cac (127.10.207.2:46849)
I20260807 08:37:44.562820 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:34065
--local_ip_for_outbound_sockets=127.10.207.3
--tserver_master_addrs=127.10.207.62:35169
--webserver_port=38045
--webserver_interface=127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab
--principal=oryx/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:44.657171 21383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:44.657403 21383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:44.657426 21383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:44.658812 21383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:44.658864 21383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:44.658893 21383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:44.660430 21383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab
--principal=oryx/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:34065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=38045
--tserver_master_addrs=127.10.207.62:35169,127.10.207.61:43817,127.10.207.60:46355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:44.660712 21383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:44.661002 21383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:44.666637 21389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.666637 21392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:44.666826 21390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:44.667052 21383 server_base.cc:1034] running on GCE node
I20260807 08:37:44.667287 21383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:44.667580 21383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:44.668761 21383 hybrid_clock.cc:648] HybridClock initialized: now 1786091864668739 us; error 42 us; skew 500 ppm
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:44.671640 21383 init.cc:377] Logged in from keytab as oryx/127.10.207.3@KRBTEST.COM (short username oryx)
I20260807 08:37:44.672194 21383 webserver.cc:460] Webserver started at http://127.10.207.3:38045/ using document root <none> and password file <none>
I20260807 08:37:44.672497 21383 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:44.672608 21383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:44.674464 21383 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:44.675266 21406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.675555 21383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:44.675787 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.672384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43082)
0807 08:37:44.672464 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.672480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.672544 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.672767 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.672768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.672771 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:44.672839 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.672841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.673741 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.673742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.674768 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.674770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.675500 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.675502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.675575 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.675581 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.675652 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.675708 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 08:37:44.675817 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.672497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.10.207.60:46355)
0807 08:37:44.672589 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.672592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.672605 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:37:44.672642 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.672643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.672670 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.672757 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:44.672923 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.673641 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.674860 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.675399 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.675486 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.675553 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.675679 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.675746 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:37:44.675666 21383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "8e27357895b849c783d74ea5b4e8edaa"
format_stamp: "Formatted at 2026-08-07 08:37:43 on dist-test-slave-h6n0"
server_key: "f937b46e61ae5b784b008431f31f14c2"
server_key_iv: "847e1da18f4f5c2e79f071d7f5740bb2"
server_key_version: "encryptionkey@0"
I20260807 08:37:44.676137 20931 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 08:37:44.676214 21383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:44.676357 20968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f518004fae84e6b951b3255e3e546a5. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.676474 20968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: This master's current role is: LEARNER
I20260807 08:37:44.690376 21383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:44.692567 21383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:44.692757 21383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:44.693027 21383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:44.693393 21383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:44.693455 21383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.693493 21383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:44.693531 21383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:44.701107 21383 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34065
I20260807 08:37:44.701259 21559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34065 every 8 connection(s)
I20260807 08:37:44.701587 21383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:44.709295 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 21383
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.61@KRBTEST.COM
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.60@KRBTEST.COM
Aug 07 08:37:44 dist-test-slave-h6n0 krb5kdc[19710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091864, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:44.717743 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.702631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:34799 (local address 127.10.207.60:46355)
0807 08:37:44.702727 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.702730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.702913 (+ 183us) server_negotiation.cc:407] Connection header received
0807 08:37:44.704347 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.704349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.704362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.704412 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.705211 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.706010 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707552 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.707839 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707904 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.712849 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.712850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.712851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.712859 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.715912 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.716477 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.716478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.716478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.716559 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.716867 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.716868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.716869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.717011 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.717088 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.717515 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.717632 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:37:44.718111 21567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.702512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.10.207.3:34799)
0807 08:37:44.702791 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.702828 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.704169 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.704459 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.704460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.704851 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.705068 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.705115 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.706154 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.706155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.707459 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.707462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.707963 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.708022 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.708881 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.708886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.712753 (+ 3867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.716025 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.716033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.716035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.716383 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.716639 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.716640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.716641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.716786 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.717120 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.717128 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.717310 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.717425 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.717642 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":139,"thread_start_us":190,"threads_started":1}
I20260807 08:37:44.719399 21560 heartbeater.cc:347] Connected to a master server at 127.10.207.60:46355
I20260807 08:37:44.719520 21560 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.719704 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.702505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.3:51733)
0807 08:37:44.702775 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.702826 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.704125 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.704452 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.704457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.704811 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.705068 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.705073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.706152 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.706154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.707459 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.707462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.707880 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.707999 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.708646 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.708657 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.713095 (+ 4438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.718399 (+ 5304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.718400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.718401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.718725 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.718950 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.718950 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.718951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.719017 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.719315 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.719316 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.719367 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.719439 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.719558 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":27,"thread_start_us":130,"threads_started":1}
I20260807 08:37:44.719775 21560 heartbeater.cc:511] Master 127.10.207.60:46355 requested a full tablet report, sending...
I20260807 08:37:44.720139 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.702848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:51733 (local address 127.10.207.62:35169)
0807 08:37:44.702901 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.702903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.702914 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:44.704276 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.704278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.704296 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.704367 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.705311 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.706030 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707593 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.707797 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707879 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.713216 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.713217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.713218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.713225 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.718312 (+ 5087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.718808 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.718810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.718810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.718874 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.719090 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.719092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.719092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.719223 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.719303 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.719916 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.720017 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:37:44.720422 20919 ts_manager.cc:194] Registered new tserver with Master: 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3:34065)
I20260807 08:37:44.720489 21561 heartbeater.cc:347] Connected to a master server at 127.10.207.62:35169
I20260807 08:37:44.720531 21561 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.720605 21561 heartbeater.cc:511] Master 127.10.207.62:35169 requested a full tablet report, sending...
I20260807 08:37:44.720635 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.703050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:52633 (local address 127.10.207.61:43817)
0807 08:37:44.703131 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.703134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.703143 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:44.704301 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.704303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.704318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.704354 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.705185 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.706024 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707408 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.707700 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707766 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.713303 (+ 5537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.713304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.713305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.713324 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.716079 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.719582 (+ 3503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.719584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.719584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.719658 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.719894 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.719895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.719895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.720055 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.720114 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.720393 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.720490 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:37:44.720856 19775 ts_manager.cc:194] Re-registered known tserver with Master: 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3:34065)
I20260807 08:37:44.721001 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.702913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.10.207.3:52633)
0807 08:37:44.703039 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.703047 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.704173 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.704459 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.704460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.704844 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:44.705068 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.705075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.706150 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.706151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.707320 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.707323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.707843 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.707844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.708003 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.708980 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:44.708984 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:44.713204 (+ 4220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:44.718430 (+ 5226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.718431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.718431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.719454 (+ 1023us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.719737 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:44.719738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:44.719738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:44.719806 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:44.720158 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:44.720160 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:44.720258 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.720719 (+ 461us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.720870 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":170,"thread_start_us":73,"threads_started":1}
I20260807 08:37:44.721428 19775 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:51733
I20260807 08:37:44.721500 21562 heartbeater.cc:347] Connected to a master server at 127.10.207.61:43817
I20260807 08:37:44.721571 21562 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:44.721659 21562 heartbeater.cc:511] Master 127.10.207.61:43817 requested a full tablet report, sending...
I20260807 08:37:44.721994 20647 ts_manager.cc:194] Registered new tserver with Master: 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3:34065)
I20260807 08:37:44.722317 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.710944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.10.207.60:46355)
0807 08:37:44.711093 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.711095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.711312 (+ 217us) server_negotiation.cc:407] Connection header received
0807 08:37:44.711347 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.711348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.711365 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.711448 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.712136 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.712842 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.715454 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.715777 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.715826 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.717333 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.717334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.717334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.717341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.719570 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.721421 (+ 1851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.721422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.721422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.721485 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.721702 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.721703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.721703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.721834 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.721908 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.722147 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.722236 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260807 08:37:44.723546 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.710448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.10.207.62:35169)
0807 08:37:44.710627 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.710629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.710824 (+ 195us) server_negotiation.cc:407] Connection header received
0807 08:37:44.710933 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.710934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.710948 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.710989 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.711885 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.712673 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.714487 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.714816 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.714860 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.716515 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.716516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.716517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.716535 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.719254 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.722182 (+ 2928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.722183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.722184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.722248 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.722584 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.722586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.722586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.722731 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.722839 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.723390 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.723484 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
I20260807 08:37:44.725728 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.712281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.10.207.61:43817)
0807 08:37:44.712406 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.712408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.712417 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:44.712439 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.712440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.712452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.712525 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.713621 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.714430 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.718636 (+ 4206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.718979 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.719056 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.722331 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.722332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.722332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.722339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.724191 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.724562 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.724563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.724563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.724636 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.724962 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.724963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.724964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.725152 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.725257 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.725467 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.725604 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260807 08:37:44.725970 19775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:44.728193 19775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:44.729103 20596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: NON_VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: true } } }
I20260807 08:37:44.729221 20596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:44.730893 21585 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: attempting to promote NON_VOTER 36b08ff551244486b2da6e53cbf0ac1d to VOTER
I20260807 08:37:44.731083 21585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } }
I20260807 08:37:44.731783 20661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Refusing update from remote peer 9f518004fae84e6b951b3255e3e546a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260807 08:37:44.731909 20930 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 LEARNER]: Refusing update from remote peer 9f518004fae84e6b951b3255e3e546a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260807 08:37:44.732070 21583 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 08:37:44.732254 20596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 08:37:44.733085 20596 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.733340 20930 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.734148 21583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.734154 20596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.734313 20596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:44.734277 20968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.734354 20968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:44.734380 20596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.734453 20596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:44.734266 21583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:44.734598 20661 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.735636 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f518004fae84e6b951b3255e3e546a5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:44.735775 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:44.739967 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.736273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:46849 (local address 127.0.0.1:37310)
0807 08:37:44.736404 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.736416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.736467 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.736854 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.736855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.736857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:44.736914 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.736916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.737769 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.737770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.738774 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.738777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.739713 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.739713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.739781 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.739785 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.739840 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.739890 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":12}
I20260807 08:37:44.740245 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.736402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.10.207.2:46849)
0807 08:37:44.736655 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.736660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.736670 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:44.736713 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.736715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.736731 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.736843 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:44.736985 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.737676 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.738873 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.739570 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.739706 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.739798 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.740096 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.740166 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":125,"threads_started":1}
I20260807 08:37:44.740901 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.736313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42489 (local address 127.0.0.1:35562)
0807 08:37:44.736454 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.736465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.736519 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.737196 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.737197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.737199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:44.737242 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.737243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.738324 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.738325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.739578 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.739580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.740659 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.740660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.740725 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.740729 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.740792 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.740829 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260807 08:37:44.740952 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.736558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.10.207.1:42489)
0807 08:37:44.737005 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.737011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.737023 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:37:44.737071 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.737074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.737090 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.737200 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:44.737300 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.738220 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.739700 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.740521 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.740641 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.740719 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.740817 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.740862 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":95,"threads_started":1}
I20260807 08:37:44.741129 21280 tablet_service.cc:1467] Processing CreateTablet for tablet e99e0aaf007f40f5a36d9958a783f48f (DEFAULT_TABLE table=test-table [id=1bed3505aaed44a4981cf25dafe740b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:44.741629 21280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e99e0aaf007f40f5a36d9958a783f48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:44.741655 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.736847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.10.207.3:34065)
0807 08:37:44.737793 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.737796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.737804 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:44.737832 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.737833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.737845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.737939 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:44.738077 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.739094 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.740581 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.741270 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.741348 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.741423 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.741547 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.741583 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":82,"threads_started":1}
I20260807 08:37:44.741691 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.736516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34065 (local address 127.0.0.1:50780)
0807 08:37:44.736671 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.736681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.736720 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.737950 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.737951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.737954 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:44.738010 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.738012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.739191 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.739192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.740292 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.740296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.741373 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.741374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.741445 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.741450 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.741521 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.741581 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":113,"threads_started":1}
I20260807 08:37:44.741832 21076 tablet_service.cc:1467] Processing CreateTablet for tablet e99e0aaf007f40f5a36d9958a783f48f (DEFAULT_TABLE table=test-table [id=1bed3505aaed44a4981cf25dafe740b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:44.742215 21076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e99e0aaf007f40f5a36d9958a783f48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:44.742466 21469 tablet_service.cc:1467] Processing CreateTablet for tablet e99e0aaf007f40f5a36d9958a783f48f (DEFAULT_TABLE table=test-table [id=1bed3505aaed44a4981cf25dafe740b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:44.742848 21469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e99e0aaf007f40f5a36d9958a783f48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:44.745117 21608 tablet_bootstrap.cc:492] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Bootstrap starting.
I20260807 08:37:44.745775 21608 tablet_bootstrap.cc:654] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:44.746088 21608 log.cc:824] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:44.746106 21609 tablet_bootstrap.cc:492] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Bootstrap starting.
I20260807 08:37:44.747076 21609 tablet_bootstrap.cc:654] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:44.747150 21611 tablet_bootstrap.cc:492] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Bootstrap starting.
I20260807 08:37:44.747490 21609 log.cc:824] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:44.747742 21608 tablet_bootstrap.cc:492] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: No bootstrap required, opened a new log
I20260807 08:37:44.747833 21608 ts_tablet_manager.cc:1397] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 08:37:44.747828 21611 tablet_bootstrap.cc:654] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:44.748202 21611 log.cc:824] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:44.748543 21609 tablet_bootstrap.cc:492] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: No bootstrap required, opened a new log
I20260807 08:37:44.748705 21609 ts_tablet_manager.cc:1397] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:37:44.748996 21611 tablet_bootstrap.cc:492] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: No bootstrap required, opened a new log
I20260807 08:37:44.749082 21611 ts_tablet_manager.cc:1397] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:44.751060 21608 raft_consensus.cc:348] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.751197 21611 raft_consensus.cc:348] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.751317 21608 raft_consensus.cc:374] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:44.751339 21611 raft_consensus.cc:374] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:44.751382 21608 raft_consensus.cc:729] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb5534a84574f759c0644363266b675, State: Initialized, Role: FOLLOWER
I20260807 08:37:44.751381 21611 raft_consensus.cc:729] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e27357895b849c783d74ea5b4e8edaa, State: Initialized, Role: FOLLOWER
I20260807 08:37:44.751585 21611 consensus_queue.cc:260] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.751569 21608 consensus_queue.cc:260] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.751906 21611 ts_tablet_manager.cc:1428] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:37:44.751909 21608 ts_tablet_manager.cc:1428] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:37:44.752038 21175 heartbeater.cc:503] Master 127.10.207.62:35169 was elected leader, sending a full tablet report...
I20260807 08:37:44.751969 21609 raft_consensus.cc:348] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.752317 21609 raft_consensus.cc:374] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:44.752388 21609 raft_consensus.cc:729] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60b419870264b5997888887b3177cac, State: Initialized, Role: FOLLOWER
I20260807 08:37:44.752599 21609 consensus_queue.cc:260] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.753023 21609 ts_tablet_manager.cc:1428] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 08:37:44.753386 21375 heartbeater.cc:503] Master 127.10.207.62:35169 was elected leader, sending a full tablet report...
I20260807 08:37:44.753491 21561 heartbeater.cc:503] Master 127.10.207.62:35169 was elected leader, sending a full tablet report...
W20260807 08:37:44.784541 21378 tablet.cc:2369] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:44.842262 21617 raft_consensus.cc:482] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:44.842422 21617 raft_consensus.cc:504] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.842963 21617 leader_election.cc:288] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60b419870264b5997888887b3177cac (127.10.207.2:46849), 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489)
I20260807 08:37:44.847505 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.843322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:53851 (local address 127.10.207.1:42489)
0807 08:37:44.843410 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.843413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.843423 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:44.843451 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.843451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.843461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.843525 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:44.843836 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.844779 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.846447 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.847059 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.847155 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.847165 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.847378 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.847420 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 08:37:44.847505 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.843234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:45623 (local address 127.10.207.2:46849)
0807 08:37:44.843281 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.843283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.843318 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:37:44.843437 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.843438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.843448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.843510 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:44.843836 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.844779 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.846447 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.847059 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.847156 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.847165 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.847379 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.847420 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:37:44.847505 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.843170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:46849 (local address 127.10.207.3:45623)
0807 08:37:44.843267 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.843280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.843333 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.843568 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.843569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.843570 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:44.843715 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.843717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.844904 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.844905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.846329 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.846333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.847182 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.847184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.847274 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.847279 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.847351 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.847406 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":66}
I20260807 08:37:44.847519 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.843222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42489 (local address 127.10.207.3:53851)
0807 08:37:44.843285 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:44.843300 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:44.843350 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:44.843562 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:44.843563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:44.843566 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:44.843715 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.843717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.844904 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.844905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.846330 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:44.846333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.847182 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.847184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:44.847274 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.847279 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:44.847351 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:37:44.847406 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":157}
I20260807 08:37:44.848013 21311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99e0aaf007f40f5a36d9958a783f48f" candidate_uuid: "8e27357895b849c783d74ea5b4e8edaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60b419870264b5997888887b3177cac" is_pre_election: true
I20260807 08:37:44.848263 21311 raft_consensus.cc:2454] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e27357895b849c783d74ea5b4e8edaa in term 0.
I20260807 08:37:44.848189 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99e0aaf007f40f5a36d9958a783f48f" candidate_uuid: "8e27357895b849c783d74ea5b4e8edaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb5534a84574f759c0644363266b675" is_pre_election: true
I20260807 08:37:44.848441 21090 raft_consensus.cc:2454] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e27357895b849c783d74ea5b4e8edaa in term 0.
I20260807 08:37:44.848570 21407 leader_election.cc:302] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e27357895b849c783d74ea5b4e8edaa, b60b419870264b5997888887b3177cac; no voters:
I20260807 08:37:44.848767 21617 raft_consensus.cc:2781] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:44.848904 21617 raft_consensus.cc:482] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:44.848946 21617 raft_consensus.cc:3037] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:44.849918 21617 raft_consensus.cc:504] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.850131 21617 leader_election.cc:288] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [CANDIDATE]: Term 1 election: Requested vote from peers b60b419870264b5997888887b3177cac (127.10.207.2:46849), 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489)
I20260807 08:37:44.850458 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99e0aaf007f40f5a36d9958a783f48f" candidate_uuid: "8e27357895b849c783d74ea5b4e8edaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb5534a84574f759c0644363266b675"
I20260807 08:37:44.850457 21311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99e0aaf007f40f5a36d9958a783f48f" candidate_uuid: "8e27357895b849c783d74ea5b4e8edaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60b419870264b5997888887b3177cac"
I20260807 08:37:44.850608 21090 raft_consensus.cc:3037] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:44.850611 21311 raft_consensus.cc:3037] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:44.851728 21311 raft_consensus.cc:2454] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e27357895b849c783d74ea5b4e8edaa in term 1.
I20260807 08:37:44.851836 21090 raft_consensus.cc:2454] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e27357895b849c783d74ea5b4e8edaa in term 1.
I20260807 08:37:44.851905 21407 leader_election.cc:302] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e27357895b849c783d74ea5b4e8edaa, b60b419870264b5997888887b3177cac; no voters:
I20260807 08:37:44.851999 21617 raft_consensus.cc:2781] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:44.852387 21617 raft_consensus.cc:686] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 1 LEADER]: Becoming Leader. State: Replica: 8e27357895b849c783d74ea5b4e8edaa, State: Running, Role: LEADER
I20260807 08:37:44.852553 21617 consensus_queue.cc:237] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } }
I20260807 08:37:44.853801 19773 catalog_manager.cc:5495] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3). New cstate: current_term: 1 leader_uuid: "8e27357895b849c783d74ea5b4e8edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } health_report { overall_health: HEALTHY } } }
W20260807 08:37:44.863619 21177 tablet.cc:2369] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:44.888152 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.885684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.10.207.3:34065)
0807 08:37:44.885758 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.885760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.885769 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:44.885807 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.885808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.885824 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.885903 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:44.886044 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.886595 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.887362 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.887562 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.887638 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.887817 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:44.887916 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:44.888014 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.888051 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.888071 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:37:44.890122 21311 raft_consensus.cc:1261] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 1 FOLLOWER]: Refusing update from remote peer 8e27357895b849c783d74ea5b4e8edaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:44.890187 21090 raft_consensus.cc:1261] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Refusing update from remote peer 8e27357895b849c783d74ea5b4e8edaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:44.890483 21617 consensus_queue.cc:1035] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:44.890627 21621 consensus_queue.cc:1035] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:44.893527 21628 mvcc.cc:205] Tried to move back new op lower bound from 7315832278587019264 to 7315832278437015552. Current Snapshot: MvccSnapshot[applied={T|T < 7315832278587019264}]
I20260807 08:37:44.893903 21629 mvcc.cc:205] Tried to move back new op lower bound from 7315832278587019264 to 7315832278437015552. Current Snapshot: MvccSnapshot[applied={T|T < 7315832278587019264}]
I20260807 08:37:44.894588 21630 mvcc.cc:205] Tried to move back new op lower bound from 7315832278587019264 to 7315832278437015552. Current Snapshot: MvccSnapshot[applied={T|T < 7315832278587019264 or (T in {7315832278587019264})}]
I20260807 08:37:44.896309 19773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:44.896416 19773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:44.898155 19773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f518004fae84e6b951b3255e3e546a5: Sending DeleteTablet for 3 replicas of tablet e99e0aaf007f40f5a36d9958a783f48f
I20260807 08:37:44.898607 21280 tablet_service.cc:1514] Processing DeleteTablet for tablet e99e0aaf007f40f5a36d9958a783f48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:44 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37310
I20260807 08:37:44.898918 21076 tablet_service.cc:1514] Processing DeleteTablet for tablet e99e0aaf007f40f5a36d9958a783f48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:44 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:35562
I20260807 08:37:44.899390 21469 tablet_service.cc:1514] Processing DeleteTablet for tablet e99e0aaf007f40f5a36d9958a783f48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:44 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:50780
I20260807 08:37:44.899452 21643 tablet_replica.cc:330] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: stopping tablet replica
I20260807 08:37:44.899477 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 19729
I20260807 08:37:44.899572 21643 raft_consensus.cc:2229] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:44.906243 21643 raft_consensus.cc:2258] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:44.907054 21644 tablet_replica.cc:330] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: stopping tablet replica
I20260807 08:37:44.907231 21644 raft_consensus.cc:2229] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:44.907401 21644 raft_consensus.cc:2258] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:44.907972 21643 ts_tablet_manager.cc:1905] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:44.907994 21644 ts_tablet_manager.cc:1905] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:44.910073 21644 ts_tablet_manager.cc:1918] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:44.910209 21643 ts_tablet_manager.cc:1918] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:44.910228 21644 log.cc:1196] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/e99e0aaf007f40f5a36d9958a783f48f
I20260807 08:37:44.910303 21643 log.cc:1196] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/e99e0aaf007f40f5a36d9958a783f48f
I20260807 08:37:44.910708 21643 ts_tablet_manager.cc:1939] T e99e0aaf007f40f5a36d9958a783f48f P 9bb5534a84574f759c0644363266b675: Deleting consensus metadata
I20260807 08:37:44.910815 21644 ts_tablet_manager.cc:1939] T e99e0aaf007f40f5a36d9958a783f48f P 8e27357895b849c783d74ea5b4e8edaa: Deleting consensus metadata
I20260807 08:37:44.914855 21640 tablet_replica.cc:330] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: stopping tablet replica
I20260807 08:37:44.915060 21640 raft_consensus.cc:2229] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:44.915242 21640 raft_consensus.cc:2258] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:44.915845 21640 ts_tablet_manager.cc:1905] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:37:44.916996 21409 connection.cc:535] server connection from 127.0.0.1:50780 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:37:44.917321 21012 connection.cc:535] server connection from 127.0.0.1:35562 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:37:44.917874 21640 ts_tablet_manager.cc:1918] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 08:37:44.917905 20900 connection.cc:535] server connection from 127.0.0.1:43082 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:37:44.918059 21640 log.cc:1196] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/e99e0aaf007f40f5a36d9958a783f48f
I20260807 08:37:44.918412 21640 ts_tablet_manager.cc:1939] T e99e0aaf007f40f5a36d9958a783f48f P b60b419870264b5997888887b3177cac: Deleting consensus metadata
W20260807 08:37:44.919730 21217 connection.cc:441] server connection from 127.0.0.1:37310 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37310 (request call id 1) could send its response
I20260807 08:37:44.923753 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.899003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43090 (local address 127.10.207.60:46355)
0807 08:37:44.899055 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.899057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.899410 (+ 353us) server_negotiation.cc:407] Connection header received
0807 08:37:44.914770 (+ 15360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.914772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.914789 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.914857 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.915741 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.916470 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.917585 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.917909 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.917955 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.920425 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.920426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.920427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.920434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.922495 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.922922 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.922923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.922923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.922980 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.923162 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.923162 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.923163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.923282 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.923356 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.923575 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.923660 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:37:44.928896 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.920235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.10.207.61:43817)
0807 08:37:44.920330 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.920332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.920447 (+ 115us) server_negotiation.cc:407] Connection header received
0807 08:37:44.920544 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.920545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.920561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.920592 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.921236 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.922085 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.923321 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.923669 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.923745 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.925734 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.925735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.925736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.925744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.927525 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.927886 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.927887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.927888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.927946 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.928212 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.928213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.928214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.928382 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.928504 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.928680 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.928789 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 08:37:44.930121 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:44.920515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.10.207.60:46355)
0807 08:37:44.920576 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:44.920580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:44.921351 (+ 771us) server_negotiation.cc:407] Connection header received
0807 08:37:44.921449 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:44.921450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:44.921460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:44.921491 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:44.923346 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.924020 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.924966 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:44.925172 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:44.925261 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:44.927024 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:44.927026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:44.927027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:44.927035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:44.928941 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.929334 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.929335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.929335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.929391 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:44.929634 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:44.929635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:44.929635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:44.929743 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:44.929846 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:44.929949 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:37:44.930027 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:37:45.895884 21657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:45.895478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.3:38223)
0807 08:37:45.895691 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:45.895820 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":116,"threads_started":1}
W20260807 08:37:45.896268 21561 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:35169 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
I20260807 08:37:45.897317 21658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:45.896967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.2:59415)
0807 08:37:45.897148 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:45.897257 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
W20260807 08:37:45.897531 21375 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:35169 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
I20260807 08:37:45.899688 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:45.899451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.1:51323)
0807 08:37:45.899573 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:45.899639 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
W20260807 08:37:45.900012 21175 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:35169 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
I20260807 08:37:46.398720 21662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9f518004fae84e6b951b3255e3e546a5)
I20260807 08:37:46.398905 21662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } }
I20260807 08:37:46.399317 21662 leader_election.cc:288] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169), cdd89981a4ae4903bf1af7630e7c02e8 (127.10.207.61:43817)
I20260807 08:37:46.399305 21663 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9f518004fae84e6b951b3255e3e546a5)
I20260807 08:37:46.399446 21663 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } }
I20260807 08:37:46.399662 21663 leader_election.cc:288] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169), 36b08ff551244486b2da6e53cbf0ac1d (127.10.207.60:46355)
I20260807 08:37:46.399811 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.399443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44626)
0807 08:37:46.399689 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.399738 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":55,"threads_started":1}
W20260807 08:37:46.400017 20895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
I20260807 08:37:46.400084 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.399744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44640)
0807 08:37:46.399984 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.400043 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":76,"threads_started":1}
W20260807 08:37:46.400184 20625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
I20260807 08:37:46.400763 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.400602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44648)
0807 08:37:46.400671 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.400717 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 08:37:46.400795 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.400685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44662)
0807 08:37:46.400737 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.400762 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 08:37:46.400866 20625 leader_election.cc:334] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169): Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
W20260807 08:37:46.400990 20895 leader_election.cc:334] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169): Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
I20260807 08:37:46.403352 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.399536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43817 (local address 127.0.0.1:45990)
0807 08:37:46.399880 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.399892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:46.400007 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:46.400262 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:46.400263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:46.400267 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:46.400357 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.400359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.401321 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.401322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.402324 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.402326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.403128 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.403129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.403176 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.403180 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:46.403229 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 08:37:46.403262 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":75,"threads_started":1}
I20260807 08:37:46.403380 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.399593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45990 (local address 127.10.207.61:43817)
0807 08:37:46.399793 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.399795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.399970 (+ 175us) server_negotiation.cc:407] Connection header received
0807 08:37:46.400102 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.400104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.400117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.400165 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:46.400438 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.401232 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.402417 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.403036 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.403122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.403135 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.403259 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.403312 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":58,"threads_started":1}
I20260807 08:37:46.403719 20661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36b08ff551244486b2da6e53cbf0ac1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" is_pre_election: true
I20260807 08:37:46.403851 20661 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36b08ff551244486b2da6e53cbf0ac1d in term 1.
I20260807 08:37:46.404060 20897 leader_election.cc:302] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36b08ff551244486b2da6e53cbf0ac1d, cdd89981a4ae4903bf1af7630e7c02e8; no voters: 9f518004fae84e6b951b3255e3e546a5
I20260807 08:37:46.404219 21662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 08:37:46.404268 21662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9f518004fae84e6b951b3255e3e546a5)
I20260807 08:37:46.404268 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.399764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:46355 (local address 127.0.0.1:43094)
0807 08:37:46.400054 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.400065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:46.400160 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:46.400812 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:46.400814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:46.400817 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:46.400893 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.400896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.402033 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.402034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.403205 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.403207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.403999 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.404000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.404066 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.404072 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:37:46.404145 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 08:37:46.404193 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":97,"threads_started":1}
I20260807 08:37:46.404304 21662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:37:46.404408 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.400351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43094 (local address 127.10.207.60:46355)
0807 08:37:46.400481 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.400483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.400493 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:46.400677 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.400678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.400691 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.401118 (+ 427us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:46.401160 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.401936 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.403318 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.403865 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.403988 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.404000 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.404194 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.404236 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260807 08:37:46.404758 20930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "36b08ff551244486b2da6e53cbf0ac1d" is_pre_election: true
I20260807 08:37:46.405367 21662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } }
I20260807 08:37:46.405548 20930 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdd89981a4ae4903bf1af7630e7c02e8 in current term 2: Already voted for candidate 36b08ff551244486b2da6e53cbf0ac1d in this term.
I20260807 08:37:46.405546 21662 leader_election.cc:288] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [CANDIDATE]: Term 2 election: Requested vote from peers 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169), cdd89981a4ae4903bf1af7630e7c02e8 (127.10.207.61:43817)
I20260807 08:37:46.405719 20661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36b08ff551244486b2da6e53cbf0ac1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cdd89981a4ae4903bf1af7630e7c02e8"
I20260807 08:37:46.405814 20661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:37:46.405843 20628 leader_election.cc:302] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdd89981a4ae4903bf1af7630e7c02e8; no voters: 36b08ff551244486b2da6e53cbf0ac1d, 9f518004fae84e6b951b3255e3e546a5
I20260807 08:37:46.405959 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.405631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44668)
0807 08:37:46.405855 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.405935 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260807 08:37:46.406289 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.406186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44676)
0807 08:37:46.406242 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.406268 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:37:46.406419 20895 leader_election.cc:334] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169): Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
I20260807 08:37:46.406780 20661 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36b08ff551244486b2da6e53cbf0ac1d in term 2.
I20260807 08:37:46.406873 21663 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:37:46.407006 20897 leader_election.cc:302] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36b08ff551244486b2da6e53cbf0ac1d, cdd89981a4ae4903bf1af7630e7c02e8; no voters: 9f518004fae84e6b951b3255e3e546a5
I20260807 08:37:46.407110 21662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:37:46.407296 21662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [term 2 LEADER]: Becoming Leader. State: Replica: 36b08ff551244486b2da6e53cbf0ac1d, State: Running, Role: LEADER
I20260807 08:37:46.407398 21662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } }
I20260807 08:37:46.407863 21677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36b08ff551244486b2da6e53cbf0ac1d. Latest consensus state: current_term: 2 leader_uuid: "36b08ff551244486b2da6e53cbf0ac1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:46.407992 21677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:46.408355 21679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:46.408962 21679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:46.409039 21679 catalog_manager.cc:1259] Loaded cluster ID: 267afb8bff0b4f77ad061cf56a20e1ec
I20260807 08:37:46.409056 21679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:46.409307 21679 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:46.409417 21679 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Loaded TSK: 0
I20260807 08:37:46.409593 21679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:37:46.417273 20919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:46.419176 20919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:46.420287 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.420162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44694)
0807 08:37:46.420226 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.420266 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:37:46.420297 20661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [term 2 FOLLOWER]: Refusing update from remote peer 36b08ff551244486b2da6e53cbf0ac1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 08:37:46.420657 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.420530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.0.0.1:44704)
0807 08:37:46.420582 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.420624 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 08:37:46.420759 20895 consensus_peers.cc:487] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d -> Peer 9f518004fae84e6b951b3255e3e546a5 (127.10.207.62:35169): Couldn't send request to peer 9f518004fae84e6b951b3255e3e546a5. Status: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:37:46.420826 21677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 08:37:46.421553 21663 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36b08ff551244486b2da6e53cbf0ac1d. Latest consensus state: current_term: 2 leader_uuid: "36b08ff551244486b2da6e53cbf0ac1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:46.421659 21663 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:46.422320 21677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "36b08ff551244486b2da6e53cbf0ac1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:46.422431 21677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:46.422549 21663 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "36b08ff551244486b2da6e53cbf0ac1d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9f518004fae84e6b951b3255e3e546a5" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 35169 } } peers { permanent_uuid: "cdd89981a4ae4903bf1af7630e7c02e8" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43817 } attrs { promote: false } } peers { permanent_uuid: "36b08ff551244486b2da6e53cbf0ac1d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 46355 } attrs { promote: false } } }
I20260807 08:37:46.422659 21663 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd89981a4ae4903bf1af7630e7c02e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:37:46.428329 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.424826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.10.207.3:34065)
0807 08:37:46.424985 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.424987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.424999 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:37:46.425027 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.425028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.425041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.425288 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:46.425326 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.425915 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.427598 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.427965 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.428065 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.428078 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.428226 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.428265 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260807 08:37:46.428315 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.424693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34065 (local address 127.0.0.1:50790)
0807 08:37:46.424832 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.424843 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:46.424897 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:46.425157 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:46.425159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:46.425162 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:46.425216 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.425218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.426021 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.426022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.427068 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.427071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.428067 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.428068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.428131 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.428135 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:46.428207 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:37:46.428250 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 08:37:46.428412 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.424580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42489 (local address 127.0.0.1:35570)
0807 08:37:46.424620 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.424632 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:46.424675 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:46.425242 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:46.425243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:46.425248 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:46.425305 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.425306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.426347 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.426348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.427358 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.427360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.428182 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.428183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.428247 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.428253 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:37:46.428285 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 08:37:46.428339 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 08:37:46.428684 21691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.424697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:46849 (local address 127.0.0.1:37320)
0807 08:37:46.425157 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.425167 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:46.425236 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:46.425445 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:46.425445 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:46.425447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:46.425484 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.425486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.426498 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.426498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.427514 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.427516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.428333 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.428334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.428513 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.428518 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:46.428563 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:37:46.428621 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":77,"thread_start_us":78,"threads_started":1}
I20260807 08:37:46.428753 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.424893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.10.207.1:42489)
0807 08:37:46.425030 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.425032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.425040 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:46.425074 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.425075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.425088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.425381 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:46.425424 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.426235 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.427451 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.428076 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.428562 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.428578 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.428611 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.428654 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260807 08:37:46.428910 21469 tablet_service.cc:1467] Processing CreateTablet for tablet 54664bfbb5c14245888593476229bdbc (DEFAULT_TABLE table=test-table [id=1303000634c24a15a17ba11ec1640747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:46.428951 21076 tablet_service.cc:1467] Processing CreateTablet for tablet 54664bfbb5c14245888593476229bdbc (DEFAULT_TABLE table=test-table [id=1303000634c24a15a17ba11ec1640747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:46.429042 21469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54664bfbb5c14245888593476229bdbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:46.429104 21076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54664bfbb5c14245888593476229bdbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:46.430303 21694 tablet_bootstrap.cc:492] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Bootstrap starting.
I20260807 08:37:46.430648 21695 tablet_bootstrap.cc:492] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675: Bootstrap starting.
I20260807 08:37:46.430797 21694 tablet_bootstrap.cc:654] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:46.430822 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.424756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.10.207.2:46849)
0807 08:37:46.425102 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.425105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.425212 (+ 107us) server_negotiation.cc:407] Connection header received
0807 08:37:46.425317 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.425318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.425330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.425382 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:46.425589 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.426410 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.427604 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.428216 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.428284 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.428300 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.428826 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.428876 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":72,"threads_started":1}
I20260807 08:37:46.431066 21280 tablet_service.cc:1467] Processing CreateTablet for tablet 54664bfbb5c14245888593476229bdbc (DEFAULT_TABLE table=test-table [id=1303000634c24a15a17ba11ec1640747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:46.431110 21695 tablet_bootstrap.cc:654] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:46.431227 21280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54664bfbb5c14245888593476229bdbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:46.431721 21694 tablet_bootstrap.cc:492] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: No bootstrap required, opened a new log
I20260807 08:37:46.431792 21694 ts_tablet_manager.cc:1397] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:37:46.431870 21695 tablet_bootstrap.cc:492] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675: No bootstrap required, opened a new log
I20260807 08:37:46.431939 21695 ts_tablet_manager.cc:1397] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:46.431965 21694 raft_consensus.cc:348] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.432065 21694 raft_consensus.cc:374] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:46.432085 21694 raft_consensus.cc:729] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e27357895b849c783d74ea5b4e8edaa, State: Initialized, Role: FOLLOWER
I20260807 08:37:46.432168 21694 consensus_queue.cc:260] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.432137 21695 raft_consensus.cc:348] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.432227 21695 raft_consensus.cc:374] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:46.432255 21695 raft_consensus.cc:729] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb5534a84574f759c0644363266b675, State: Initialized, Role: FOLLOWER
I20260807 08:37:46.432296 21695 consensus_queue.cc:260] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.432355 21694 ts_tablet_manager.cc:1428] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:46.432731 21695 ts_tablet_manager.cc:1428] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:46.433171 21704 tablet_bootstrap.cc:492] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: Bootstrap starting.
I20260807 08:37:46.433653 21704 tablet_bootstrap.cc:654] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:46.433986 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.433474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.1:55235)
0807 08:37:46.433639 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.433700 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260807 08:37:46.434099 21174 heartbeater.cc:503] Master 127.10.207.60:46355 was elected leader, sending a full tablet report...
I20260807 08:37:46.434707 21704 tablet_bootstrap.cc:492] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: No bootstrap required, opened a new log
I20260807 08:37:46.434762 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.432580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.3:54165)
0807 08:37:46.434660 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.434725 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2050,"thread_start_us":84,"threads_started":1}
I20260807 08:37:46.434764 21704 ts_tablet_manager.cc:1397] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:46.434928 21704 raft_consensus.cc:348] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.435015 21704 raft_consensus.cc:374] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:46.435051 21704 raft_consensus.cc:729] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60b419870264b5997888887b3177cac, State: Initialized, Role: FOLLOWER
I20260807 08:37:46.435137 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.434999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.3:59921)
0807 08:37:46.435068 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.435105 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 08:37:46.435122 21704 consensus_queue.cc:260] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.435333 21704 ts_tablet_manager.cc:1428] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:46.435806 21710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.435600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.2:37093)
0807 08:37:46.435734 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.435772 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":90,"threads_started":1}
I20260807 08:37:46.435927 21374 heartbeater.cc:503] Master 127.10.207.60:46355 was elected leader, sending a full tablet report...
W20260807 08:37:46.453537 21564 tablet.cc:2369] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:46.492698 21707 raft_consensus.cc:482] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:46.492803 21707 raft_consensus.cc:504] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.493227 21707 leader_election.cc:288] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489), 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3:34065)
I20260807 08:37:46.496584 21710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.493380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42489 (local address 127.10.207.2:59099)
0807 08:37:46.493448 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.493460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:46.493515 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:46.493769 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:46.493770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:46.493773 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:46.493879 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.493881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.494709 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.494710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.495628 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.495630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.496300 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.496301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.496390 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.496397 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:37:46.496465 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:37:46.496513 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":41}
I20260807 08:37:46.496670 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.493517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:59099 (local address 127.10.207.1:42489)
0807 08:37:46.493591 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.493594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.493606 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:37:46.493643 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.493644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.493658 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.493739 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:46.493993 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.494619 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.495730 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.496206 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.496269 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.496280 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.496510 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.496588 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:37:46.496831 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.493633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:37333 (local address 127.10.207.3:34065)
0807 08:37:46.493690 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.493692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.493833 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:37:46.493889 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.493891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.493901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.493972 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:46.494206 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.494811 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.495928 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.496340 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.496420 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.496431 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.496654 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.496744 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:37:46.496933 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.493519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34065 (local address 127.10.207.2:37333)
0807 08:37:46.493731 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.493746 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:46.493830 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:46.493995 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:46.493996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:46.493998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:46.494084 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.494085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.494911 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.494912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.495829 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:46.495831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.496482 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.496483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:46.496549 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.496553 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:46.496780 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 08:37:46.496832 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":126,"threads_started":1}
I20260807 08:37:46.497128 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54664bfbb5c14245888593476229bdbc" candidate_uuid: "b60b419870264b5997888887b3177cac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb5534a84574f759c0644363266b675" is_pre_election: true
I20260807 08:37:46.497262 21090 raft_consensus.cc:2454] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60b419870264b5997888887b3177cac in term 0.
I20260807 08:37:46.497219 21498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54664bfbb5c14245888593476229bdbc" candidate_uuid: "b60b419870264b5997888887b3177cac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e27357895b849c783d74ea5b4e8edaa" is_pre_election: true
I20260807 08:37:46.497373 21498 raft_consensus.cc:2454] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60b419870264b5997888887b3177cac in term 0.
I20260807 08:37:46.497520 21220 leader_election.cc:302] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb5534a84574f759c0644363266b675, b60b419870264b5997888887b3177cac; no voters:
I20260807 08:37:46.497741 21707 raft_consensus.cc:2781] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:46.497793 21707 raft_consensus.cc:482] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:46.497815 21707 raft_consensus.cc:3037] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:46.498778 21707 raft_consensus.cc:504] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.498929 21707 leader_election.cc:288] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [CANDIDATE]: Term 1 election: Requested vote from peers 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489), 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3:34065)
I20260807 08:37:46.499141 21498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54664bfbb5c14245888593476229bdbc" candidate_uuid: "b60b419870264b5997888887b3177cac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e27357895b849c783d74ea5b4e8edaa"
I20260807 08:37:46.499241 21498 raft_consensus.cc:3037] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:46.499195 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54664bfbb5c14245888593476229bdbc" candidate_uuid: "b60b419870264b5997888887b3177cac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb5534a84574f759c0644363266b675"
I20260807 08:37:46.499315 21090 raft_consensus.cc:3037] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:46.500030 21090 raft_consensus.cc:2454] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60b419870264b5997888887b3177cac in term 1.
I20260807 08:37:46.500061 21498 raft_consensus.cc:2454] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60b419870264b5997888887b3177cac in term 1.
I20260807 08:37:46.500232 21220 leader_election.cc:302] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e27357895b849c783d74ea5b4e8edaa, b60b419870264b5997888887b3177cac; no voters:
I20260807 08:37:46.500325 21707 raft_consensus.cc:2781] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:46.500512 21707 raft_consensus.cc:686] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 1 LEADER]: Becoming Leader. State: Replica: b60b419870264b5997888887b3177cac, State: Running, Role: LEADER
I20260807 08:37:46.500615 21707 consensus_queue.cc:237] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } }
I20260807 08:37:46.500864 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.500742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.2:57859)
0807 08:37:46.500781 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.500835 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260807 08:37:46.501511 20918 catalog_manager.cc:5495] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac reported cstate change: term changed from 0 to 1, leader changed from <none> to b60b419870264b5997888887b3177cac (127.10.207.2). New cstate: current_term: 1 leader_uuid: "b60b419870264b5997888887b3177cac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b60b419870264b5997888887b3177cac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 46849 } health_report { overall_health: HEALTHY } } }
I20260807 08:37:46.520677 21690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.518323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37326 (local address 127.10.207.2:46849)
0807 08:37:46.518387 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:46.518394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:46.518402 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:46.518436 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:46.518437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:46.518457 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:46.518601 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:46.518715 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.519194 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.519881 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:46.520070 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:46.520138 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:46.520308 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:46.520423 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:46.520487 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:46.520544 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 08:37:46.520563 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:37:46.522085 21090 raft_consensus.cc:1261] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Refusing update from remote peer b60b419870264b5997888887b3177cac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:46.522185 21498 raft_consensus.cc:1261] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 1 FOLLOWER]: Refusing update from remote peer b60b419870264b5997888887b3177cac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:46.522462 21713 consensus_queue.cc:1035] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bb5534a84574f759c0644363266b675" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:46.522653 21707 consensus_queue.cc:1035] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e27357895b849c783d74ea5b4e8edaa" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:46.523490 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.523379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.1:44891)
0807 08:37:46.523431 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.523462 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 08:37:46.524205 21560 heartbeater.cc:503] Master 127.10.207.60:46355 was elected leader, sending a full tablet report...
I20260807 08:37:46.524467 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.524342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.3:53653)
0807 08:37:46.524407 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.524435 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 08:37:46.524945 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.524854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.2:58971)
0807 08:37:46.524895 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.524920 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 08:37:46.525420 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.525320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.1:41797)
0807 08:37:46.525367 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.525392 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 08:37:46.526147 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.526051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.2:35251)
0807 08:37:46.526094 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.526120 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:37:46.526783 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.526661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:35169 (local address 127.10.207.3:41581)
0807 08:37:46.526723 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.526757 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:35169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:37:46.527796 20914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:46.527920 20914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:46.529448 20914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36b08ff551244486b2da6e53cbf0ac1d: Sending DeleteTablet for 3 replicas of tablet 54664bfbb5c14245888593476229bdbc
I20260807 08:37:46.529785 21469 tablet_service.cc:1514] Processing DeleteTablet for tablet 54664bfbb5c14245888593476229bdbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:46 UTC) from {username='oryx', principal='oryx/127.10.207.60@KRBTEST.COM'} at 127.0.0.1:50790
I20260807 08:37:46.529790 21280 tablet_service.cc:1514] Processing DeleteTablet for tablet 54664bfbb5c14245888593476229bdbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:46 UTC) from {username='oryx', principal='oryx/127.10.207.60@KRBTEST.COM'} at 127.0.0.1:37320
I20260807 08:37:46.529825 21279 tablet_service.cc:1514] Processing DeleteTablet for tablet 54664bfbb5c14245888593476229bdbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:46 UTC) from {username='oryx', principal='oryx/127.10.207.60@KRBTEST.COM'} at 127.0.0.1:37320
I20260807 08:37:46.530139 21731 tablet_replica.cc:330] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: stopping tablet replica
I20260807 08:37:46.530114 21076 tablet_service.cc:1514] Processing DeleteTablet for tablet 54664bfbb5c14245888593476229bdbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:46 UTC) from {username='oryx', principal='oryx/127.10.207.60@KRBTEST.COM'} at 127.0.0.1:35570
I20260807 08:37:46.530318 21733 tablet_replica.cc:330] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675: stopping tablet replica
I20260807 08:37:46.530372 21733 raft_consensus.cc:2229] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:46.530432 21733 raft_consensus.cc:2258] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:46.530645 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 20863
I20260807 08:37:46.530836 21733 ts_tablet_manager.cc:1905] T 54664bfbb5c14245888593476229bdbc P 9bb5534a84574f759c0644363266b675: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:46.535909 21731 raft_consensus.cc:2229] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:46.536012 21731 raft_consensus.cc:2258] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:46.536141 21732 tablet_replica.cc:330] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: stopping tablet replica
I20260807 08:37:46.536203 21732 raft_consensus.cc:2229] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:46.536455 21732 raft_consensus.cc:2258] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:46.536604 21731 ts_tablet_manager.cc:1905] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:46.538964 21731 ts_tablet_manager.cc:1918] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:46.539085 21731 log.cc:1196] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/54664bfbb5c14245888593476229bdbc
I20260807 08:37:46.539425 21731 ts_tablet_manager.cc:1939] T 54664bfbb5c14245888593476229bdbc P 8e27357895b849c783d74ea5b4e8edaa: Deleting consensus metadata
I20260807 08:37:46.539841 20900 catalog_manager.cc:4841] TS 8e27357895b849c783d74ea5b4e8edaa (127.10.207.3:34065): tablet 54664bfbb5c14245888593476229bdbc (table test-table [id=1303000634c24a15a17ba11ec1640747]) successfully deleted
I20260807 08:37:46.540406 21732 ts_tablet_manager.cc:1905] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:46.542819 21732 ts_tablet_manager.cc:1918] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:46.542901 21732 log.cc:1196] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/54664bfbb5c14245888593476229bdbc
I20260807 08:37:46.543166 21732 ts_tablet_manager.cc:1939] T 54664bfbb5c14245888593476229bdbc P b60b419870264b5997888887b3177cac: Deleting consensus metadata
I20260807 08:37:46.543653 20895 catalog_manager.cc:4841] TS b60b419870264b5997888887b3177cac (127.10.207.2:46849): tablet 54664bfbb5c14245888593476229bdbc (table test-table [id=1303000634c24a15a17ba11ec1640747]) successfully deleted
W20260807 08:37:46.543937 20895 catalog_manager.cc:4808] TS b60b419870264b5997888887b3177cac (127.10.207.2:46849): delete failed for tablet 54664bfbb5c14245888593476229bdbc because the tablet was not found. No further retry: Not found: Tablet not found: 54664bfbb5c14245888593476229bdbc
I20260807 08:37:46.547077 21691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:46.546918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42489 (local address 127.0.0.1:35572)
0807 08:37:46.546989 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:46.547030 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 08:37:46.547323 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 21187
W20260807 08:37:46.547302 20900 catalog_manager.cc:4584] TS 9bb5534a84574f759c0644363266b675 (127.10.207.1:42489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54664bfbb5c14245888593476229bdbc: Network error: Client connection negotiation failed: client connection to 127.10.207.1:42489: connect: Connection refused (error 111)
I20260807 08:37:46.560174 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 21383
I20260807 08:37:46.578390 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 20458
I20260807 08:37:46.595002 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 20725
2026-08-07T08:37:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5229 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:46 dist-test-slave-h6n0 krb5kdc[21736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:46 dist-test-slave-h6n0 krb5kdc[21736](info): set up 2 sockets
Aug 07 08:37:46 dist-test-slave-h6n0 krb5kdc[21736](info): commencing operation
krb5kdc: starting...
W20260807 08:37:47.878178 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:47 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab.
Entry for principal oryx/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab.
I20260807 08:37:48.013284 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:42723
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab
--principal=oryx/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:42723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:48.106325 21754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:48.106642 21754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:48.106684 21754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:48.107846 21754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:48.107925 21754 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:37:48.107954 21754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:48.107987 21754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:48.108017 21754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:48.108057 21754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:48.109661 21754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:42723
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.62.keytab
--principal=oryx/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:42723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:48.109973 21754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:48.110239 21754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:48.115598 21771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.115598 21766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.115659 21767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:48.115921 21754 server_base.cc:1034] running on GCE node
I20260807 08:37:48.116214 21754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:48.116593 21754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:48.117767 21754 hybrid_clock.cc:648] HybridClock initialized: now 1786091868117749 us; error 54 us; skew 500 ppm
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:48.120575 21754 init.cc:377] Logged in from keytab as oryx/127.10.207.62@KRBTEST.COM (short username oryx)
I20260807 08:37:48.121038 21754 webserver.cc:460] Webserver started at http://127.10.207.62:37081/ using document root <none> and password file <none>
I20260807 08:37:48.121300 21754 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:48.121369 21754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:48.121517 21754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:48.122714 21754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "2dbebe4088104f54a39d5772186b3660"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "eb052277f2887473ee29f69d2fc61cfc"
server_key_iv: "a53a4410dcdeebb9ab6c067a615afb55"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.123085 21754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "2dbebe4088104f54a39d5772186b3660"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "eb052277f2887473ee29f69d2fc61cfc"
server_key_iv: "a53a4410dcdeebb9ab6c067a615afb55"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.124476 21754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:37:48.125288 21781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:48.125499 21754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:48.125619 21754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "2dbebe4088104f54a39d5772186b3660"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "eb052277f2887473ee29f69d2fc61cfc"
server_key_iv: "a53a4410dcdeebb9ab6c067a615afb55"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.125694 21754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:48.143785 21754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:48.146019 21754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:48.146178 21754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:48.151528 21754 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:42723
I20260807 08:37:48.151712 21879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:42723 every 8 connection(s)
I20260807 08:37:48.151962 21754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:48.152845 21880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:48.155762 21880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Bootstrap starting.
I20260807 08:37:48.156473 21880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:48.156765 21880 log.cc:824] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:48.157567 21880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: No bootstrap required, opened a new log
I20260807 08:37:48.159593 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 21754
I20260807 08:37:48.159713 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:48.159938 11068 external_mini_cluster.cc:1428] Setting key c12f085dd8a25e59c403dcb705ec36d6
I20260807 08:37:48.160136 21880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbebe4088104f54a39d5772186b3660" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 42723 } }
I20260807 08:37:48.160363 21880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:48.160422 21880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbebe4088104f54a39d5772186b3660, State: Initialized, Role: FOLLOWER
I20260807 08:37:48.160570 21880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbebe4088104f54a39d5772186b3660" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 42723 } }
I20260807 08:37:48.160674 21880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:48.160729 21880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:48.160778 21880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:48.161586 21880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbebe4088104f54a39d5772186b3660" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 42723 } }
I20260807 08:37:48.161731 21880 leader_election.cc:302] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dbebe4088104f54a39d5772186b3660; no voters:
I20260807 08:37:48.161942 21880 leader_election.cc:288] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:48.162199 21892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:48.162272 21880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:48.162402 21892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [term 1 LEADER]: Becoming Leader. State: Replica: 2dbebe4088104f54a39d5772186b3660, State: Running, Role: LEADER
I20260807 08:37:48.162559 21892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbebe4088104f54a39d5772186b3660" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 42723 } }
I20260807 08:37:48.162984 21894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dbebe4088104f54a39d5772186b3660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbebe4088104f54a39d5772186b3660" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 42723 } } }
I20260807 08:37:48.163093 21894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:48.162973 21892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dbebe4088104f54a39d5772186b3660. Latest consensus state: current_term: 1 leader_uuid: "2dbebe4088104f54a39d5772186b3660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbebe4088104f54a39d5772186b3660" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 42723 } } }
I20260807 08:37:48.163326 21892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660 [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:48.163496 21903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:48.164554 21903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:48.167073 21903 catalog_manager.cc:1347] Generated new cluster ID: a18feb419d7b4c2dbe10b49be9b1a4eb
I20260807 08:37:48.167143 21903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:48.172634 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.160604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34498 (local address 127.10.207.62:42723)
0807 08:37:48.160846 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.160855 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.160893 (+ 38us) server_negotiation.cc:407] Connection header received
0807 08:37:48.161016 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.161021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.161064 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.161186 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.162168 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.164062 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.165171 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.165739 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.165885 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.168629 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.168633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.168634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.168645 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.171023 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.171403 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.171409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.171417 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.171524 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.171738 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.171739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.171740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.172063 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.172244 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.172313 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.172476 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":118,"threads_started":1}
I20260807 08:37:48.174826 21903 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:48.175463 21903 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:48.184120 21903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Generated new TSK 0
I20260807 08:37:48.184314 21903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab.
Entry for principal oryx/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab.
I20260807 08:37:48.219342 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:42723
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab
--principal=oryx/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:48.314057 21923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:48.314280 21923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:48.314321 21923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:48.315539 21923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:48.315600 21923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:48.315639 21923 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:48.315697 21923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:48.317222 21923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab
--principal=oryx/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:42723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:48.317497 21923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:48.317741 21923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:48.323118 21933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.323123 21928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.323232 21929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:48.323734 21923 server_base.cc:1034] running on GCE node
I20260807 08:37:48.323977 21923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:48.324249 21923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:48.325485 21923 hybrid_clock.cc:648] HybridClock initialized: now 1786091868325469 us; error 49 us; skew 500 ppm
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:48.328683 21923 init.cc:377] Logged in from keytab as oryx/127.10.207.1@KRBTEST.COM (short username oryx)
I20260807 08:37:48.329337 21923 webserver.cc:460] Webserver started at http://127.10.207.1:33163/ using document root <none> and password file <none>
I20260807 08:37:48.329643 21923 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:48.329715 21923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:48.330016 21923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:48.331218 21923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "7aa0cec708264197bd993cbed085f442"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "1ac17def19fdece2145aab74abce91d8"
server_key_iv: "fdf8fec6b7cd47b161e2e2051e1eced9"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.331727 21923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "7aa0cec708264197bd993cbed085f442"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "1ac17def19fdece2145aab74abce91d8"
server_key_iv: "fdf8fec6b7cd47b161e2e2051e1eced9"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.333318 21923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:48.334322 21947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:48.334700 21923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:48.334826 21923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "7aa0cec708264197bd993cbed085f442"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "1ac17def19fdece2145aab74abce91d8"
server_key_iv: "fdf8fec6b7cd47b161e2e2051e1eced9"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.334911 21923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:48.352939 21923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:48.355798 21923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:48.356011 21923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:48.356700 21923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:48.356758 21923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:48.356799 21923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:48.356813 21923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:48.366169 21923 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:42709
I20260807 08:37:48.366261 22135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:42709 every 8 connection(s)
I20260807 08:37:48.366603 21923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:48.369177 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.357249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:55523 (local address 127.10.207.62:42723)
0807 08:37:48.357336 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.357339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.357666 (+ 327us) server_negotiation.cc:407] Connection header received
0807 08:37:48.358305 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.358307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.358325 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.358431 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.359295 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.359864 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.361635 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.362052 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.362157 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.366175 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.366177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.366177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.366191 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.367895 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.368345 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.368345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.368346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.368403 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.368643 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.368644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.368644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.368751 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.368807 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.369045 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.369113 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:37:48.369453 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.357219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:42723 (local address 127.10.207.1:55523)
0807 08:37:48.357494 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.357541 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.358156 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.358585 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.358591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.358939 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:48.359174 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.359180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.359981 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.359982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.361521 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.361525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.362179 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.362180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.362423 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.363146 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:48.363158 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:48.366041 (+ 2883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:48.367990 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.367997 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.367999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.368264 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.368464 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.368465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.368465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.368579 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.368799 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:48.368802 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:48.368925 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.369010 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.369154 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":186,"threads_started":1}
I20260807 08:37:48.370072 22136 heartbeater.cc:347] Connected to a master server at 127.10.207.62:42723
I20260807 08:37:48.370172 22136 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:48.370378 22136 heartbeater.cc:511] Master 127.10.207.62:42723 requested a full tablet report, sending...
I20260807 08:37:48.371160 21805 ts_manager.cc:194] Registered new tserver with Master: 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709)
I20260807 08:37:48.372202 21805 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:55523
I20260807 08:37:48.376806 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 21923
I20260807 08:37:48.376935 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:48.377209 11068 external_mini_cluster.cc:1428] Setting key 30eb57c533d7c6c83e70815e81e4bbf2
I20260807 08:37:48.378006 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.371523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:39311 (local address 127.10.207.62:42723)
0807 08:37:48.371597 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.371599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.371716 (+ 117us) server_negotiation.cc:407] Connection header received
0807 08:37:48.371832 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.371833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.371843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.371915 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.372467 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.372983 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.373734 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.373908 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.373964 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.375058 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.375059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.375060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.375065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.376348 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.376822 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.376825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.376825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.376934 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.377310 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.377312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.377313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.377459 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.377572 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.377767 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.377894 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:37:48.378029 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.371446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:42723 (local address 127.10.207.1:39311)
0807 08:37:48.371641 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.371655 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.371732 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.371941 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.371942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.372186 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:48.372260 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.372262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.373096 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.373097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.373653 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.373655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.374009 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.374010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.374108 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.374640 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:48.374646 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:48.374973 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:48.376445 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.376446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.376447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.376693 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.377076 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.377077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.377078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.377173 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.377590 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:48.377591 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:48.377649 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.377733 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.377903 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for oryx/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab.
Entry for principal oryx/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab.
I20260807 08:37:48.405764 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:42723
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab
--principal=oryx/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:48.494433 22149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:48.494680 22149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:48.494704 22149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:48.495772 22149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:48.495810 22149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:48.495831 22149 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:48.495847 22149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:48.497268 22149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab
--principal=oryx/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:42723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:48.497443 22149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:48.497617 22149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:48.502729 22157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.502748 22158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.502785 22164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:48.503145 22149 server_base.cc:1034] running on GCE node
I20260807 08:37:48.503430 22149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:48.503722 22149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:48.504909 22149 hybrid_clock.cc:648] HybridClock initialized: now 1786091868504898 us; error 44 us; skew 500 ppm
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:48.507947 22149 init.cc:377] Logged in from keytab as oryx/127.10.207.2@KRBTEST.COM (short username oryx)
I20260807 08:37:48.508448 22149 webserver.cc:460] Webserver started at http://127.10.207.2:34681/ using document root <none> and password file <none>
I20260807 08:37:48.508718 22149 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:48.508773 22149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:48.508917 22149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:48.510134 22149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "213d0af2d18649d0afaf6b08f283d9c9"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "b484556285b685b67fc7455de0616940"
server_key_iv: "36e7df69c7015f1d1698c6db31822577"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.510601 22149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "213d0af2d18649d0afaf6b08f283d9c9"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "b484556285b685b67fc7455de0616940"
server_key_iv: "36e7df69c7015f1d1698c6db31822577"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.512149 22149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:48.513024 22173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:48.513245 22149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:48.513334 22149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "213d0af2d18649d0afaf6b08f283d9c9"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "b484556285b685b67fc7455de0616940"
server_key_iv: "36e7df69c7015f1d1698c6db31822577"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.513391 22149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:48.523531 22149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:48.525592 22149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:48.525710 22149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:48.526213 22149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:48.526262 22149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:48.526284 22149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:48.526298 22149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:48.536056 22149 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:38545
I20260807 08:37:48.536084 22359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:38545 every 8 connection(s)
I20260807 08:37:48.536582 22149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:48.538614 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.526812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:53073 (local address 127.10.207.62:42723)
0807 08:37:48.526904 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.526907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.527204 (+ 297us) server_negotiation.cc:407] Connection header received
0807 08:37:48.528162 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.528164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.528177 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.528251 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.529503 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.530030 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.531931 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.532232 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.532315 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.535177 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.535180 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.535181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.535192 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.537131 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.537509 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.537510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.537510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.537582 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.537842 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.537843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.537843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.537973 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.538063 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.538399 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.538474 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:37:48.538751 22183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.526775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:42723 (local address 127.10.207.2:53073)
0807 08:37:48.527021 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.527078 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.527874 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.528378 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.528384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.528938 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:48.529378 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.529386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.530119 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.530121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.531817 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.531821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.532335 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.532337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.532487 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.533051 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:48.533062 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:48.535072 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:48.537214 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.537218 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.537220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.537432 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.537652 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.537653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.537653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.537750 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.538068 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:48.538074 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:48.538208 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.538337 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.538486 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":124,"threads_started":1}
I20260807 08:37:48.539353 22360 heartbeater.cc:347] Connected to a master server at 127.10.207.62:42723
I20260807 08:37:48.539443 22360 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:48.539649 22360 heartbeater.cc:511] Master 127.10.207.62:42723 requested a full tablet report, sending...
I20260807 08:37:48.540158 21804 ts_manager.cc:194] Registered new tserver with Master: 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545)
I20260807 08:37:48.540584 21804 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:53073
I20260807 08:37:48.542392 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 22149
I20260807 08:37:48.542551 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:48.542799 11068 external_mini_cluster.cc:1428] Setting key 9eae7f48af9caf9c55ed6f77ca4b436a
I20260807 08:37:48.547551 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.540889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:39077 (local address 127.10.207.62:42723)
0807 08:37:48.540965 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.540967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.541067 (+ 100us) server_negotiation.cc:407] Connection header received
0807 08:37:48.541133 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.541134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.541144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.541196 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.541714 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.542198 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.543301 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.543557 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.543635 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.544739 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.544741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.544741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.544749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.546301 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.546727 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.546728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.546729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.546792 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.546992 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.546993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.546993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.547150 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.547230 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.547386 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.547464 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:37:48.547567 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.540855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:42723 (local address 127.10.207.2:39077)
0807 08:37:48.540987 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.540999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.541065 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.541205 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.541206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.541550 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:48.541621 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.541623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.542308 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.542309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.543198 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.543200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.543648 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.543649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.543717 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.544227 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:48.544233 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:48.544624 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:48.546389 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.546390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.546391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.546649 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.546859 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.546860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.546860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.546919 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.547222 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:48.547223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:48.547269 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.547359 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.547465 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab.
Entry for principal oryx/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab.
I20260807 08:37:48.571614 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:42723
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab
--principal=oryx/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:48.662515 22369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:48.662844 22369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:48.662906 22369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:48.664450 22369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:48.664499 22369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:48.664521 22369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:37:48.664541 22369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:48.666225 22369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab
--principal=oryx/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.207.62:42723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:48.666518 22369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:48.666865 22369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:48.672628 22376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.672770 22377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:48.672644 22380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:48.673080 22369 server_base.cc:1034] running on GCE node
I20260807 08:37:48.673385 22369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:48.673662 22369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:48.674873 22369 hybrid_clock.cc:648] HybridClock initialized: now 1786091868674853 us; error 57 us; skew 500 ppm
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:48.677948 22369 init.cc:377] Logged in from keytab as oryx/127.10.207.3@KRBTEST.COM (short username oryx)
I20260807 08:37:48.678445 22369 webserver.cc:460] Webserver started at http://127.10.207.3:43175/ using document root <none> and password file <none>
I20260807 08:37:48.678745 22369 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:48.678828 22369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:48.678987 22369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:48.680154 22369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "181852bc61e24d4387a01f2b9527663a"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "839abbe03f1ce1a5ba0f13e336144a6f"
server_key_iv: "07e7c85889600cde4c9776611850aea5"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.680574 22369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "181852bc61e24d4387a01f2b9527663a"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "839abbe03f1ce1a5ba0f13e336144a6f"
server_key_iv: "07e7c85889600cde4c9776611850aea5"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.682034 22369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:48.682991 22393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:48.683252 22369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:48.683367 22369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "181852bc61e24d4387a01f2b9527663a"
format_stamp: "Formatted at 2026-08-07 08:37:48 on dist-test-slave-h6n0"
server_key: "839abbe03f1ce1a5ba0f13e336144a6f"
server_key_iv: "07e7c85889600cde4c9776611850aea5"
server_key_version: "encryptionkey@0"
I20260807 08:37:48.683450 22369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:48.705788 22369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:48.707945 22369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:48.708137 22369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:48.708679 22369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:48.708808 22369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:48.708913 22369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:48.708938 22369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:48.717550 22369 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:39141
I20260807 08:37:48.717677 22550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:39141 every 8 connection(s)
I20260807 08:37:48.718077 22369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:48.721194 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.709495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:47961 (local address 127.10.207.62:42723)
0807 08:37:48.709591 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.709593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.709843 (+ 250us) server_negotiation.cc:407] Connection header received
0807 08:37:48.710455 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.710457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.710472 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.710606 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.711409 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.711961 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.713202 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.713414 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.713506 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.717913 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.717915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.717915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.717924 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.719683 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.720186 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.720187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.720187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.720245 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.720484 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.720485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.720485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.720630 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.720690 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.721043 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.721121 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:37:48.721418 22407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.709438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:42723 (local address 127.10.207.3:47961)
0807 08:37:48.709678 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.709725 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.710350 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.710719 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.710724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.711036 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:48.711294 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.711299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.712069 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.712070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.713097 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.713100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.713517 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.713518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.713749 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.714461 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:48.714474 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:48.717784 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:48.719802 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.719810 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.719813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.720096 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.720314 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.720314 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.720314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.720418 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.720733 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:48.720740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:48.720895 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.721017 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.721177 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":152,"threads_started":1}
I20260807 08:37:48.722118 22551 heartbeater.cc:347] Connected to a master server at 127.10.207.62:42723
I20260807 08:37:48.722221 22551 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:48.722447 22551 heartbeater.cc:511] Master 127.10.207.62:42723 requested a full tablet report, sending...
I20260807 08:37:48.723006 21805 ts_manager.cc:194] Registered new tserver with Master: 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:48.723443 21805 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:47961
I20260807 08:37:48.728340 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 22369
I20260807 08:37:48.728461 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:48.728660 11068 external_mini_cluster.cc:1428] Setting key a9b091ca1536cb8f902539c91c3e6045
I20260807 08:37:48.729878 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:37:48.730050 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.723838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:40531 (local address 127.10.207.62:42723)
0807 08:37:48.723887 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.723889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.724434 (+ 545us) server_negotiation.cc:407] Connection header received
0807 08:37:48.724461 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.724461 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.724471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.724525 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.724801 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.725298 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.726019 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.726235 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.726303 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.727226 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.727228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.727228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.727234 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.728637 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.729040 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.729042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.729042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.729121 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.729422 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.729423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.729423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.729571 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.729634 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.729874 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.729972 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:37:48.730137 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.723734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:42723 (local address 127.10.207.3:40531)
0807 08:37:48.724347 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.724358 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.724423 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.724530 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.724531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.724684 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:48.724734 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.724736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.725398 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.725399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.725929 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.725930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.726315 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.726316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.726366 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.726863 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:48.726868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:48.727136 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:48.728725 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.728726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.728727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.728950 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.729210 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:48.729211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:48.729211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:48.729319 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:48.729663 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:48.729664 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:48.729723 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.729849 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.730013 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":106,"threads_started":1}
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:37:48.755841 21888 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:48.749467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34512 (local address 127.10.207.62:42723)
0807 08:37:48.749538 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.749540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.749735 (+ 195us) server_negotiation.cc:407] Connection header received
0807 08:37:48.749774 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.749775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.749790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.749840 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.750523 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.751098 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.752079 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.752272 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.752355 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.755786 (+ 3431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34512: BlockingRecv error: recv got EOF from 127.0.0.1:34512 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 08:37:48 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:48.765090 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.756940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.10.207.62:42723)
0807 08:37:48.757023 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.757025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.757094 (+ 69us) server_negotiation.cc:407] Connection header received
0807 08:37:48.757133 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.757134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.757145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.757211 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:48.757903 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.758405 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.759322 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.759511 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.759579 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.761895 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:48.761896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:48.761896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:48.761903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:48.763879 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.764294 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.764295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.764296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.764354 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:48.764623 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:48.764624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:48.764624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:48.764743 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:48.764834 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.764952 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.765020 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:37:48.767222 21805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:48.769246 21805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:48.776950 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.772212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:39141 (local address 127.0.0.1:53188)
0807 08:37:48.772563 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.772574 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.772629 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.773139 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.773140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.773143 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:48.773223 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.773226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.774774 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.774775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.775679 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.775682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.776573 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.776573 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.776676 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.776692 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.776795 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.776865 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":34,"thread_start_us":59,"threads_started":1}
I20260807 08:37:48.776955 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.772503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.10.207.3:39141)
0807 08:37:48.772955 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.772961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.772971 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:48.773025 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.773028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.773045 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.773140 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:48.773335 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.774602 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.775794 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.776455 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.776568 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.776646 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.776814 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.776856 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":175,"threads_started":1}
I20260807 08:37:48.777064 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.772542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.10.207.1:42709)
0807 08:37:48.772866 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.772874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.772890 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:37:48.772948 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.772950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.772968 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.773094 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:48.773279 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.774141 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.775749 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.776473 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.776553 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.776641 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.776879 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.776936 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":100,"threads_started":1}
I20260807 08:37:48.777263 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.772208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:38545 (local address 127.0.0.1:46746)
0807 08:37:48.772351 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.772401 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.772511 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.773432 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.773433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.773435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:48.773488 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.773489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.774775 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.774777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.776071 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.776073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.776996 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.776997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.777069 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.777074 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.777104 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.777159 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":157,"thread_start_us":97,"threads_started":1}
I20260807 08:37:48.777738 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.772812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.10.207.2:38545)
0807 08:37:48.773242 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:48.773247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:48.773257 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:48.773301 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:48.773303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:48.773321 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:48.773411 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:48.773585 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.774668 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.776170 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.776900 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.777458 (+ 558us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.777548 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:48.777576 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 08:37:48.777625 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":107,"threads_started":1}
I20260807 08:37:48.777882 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:48.772417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42709 (local address 127.0.0.1:53738)
0807 08:37:48.772735 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:48.772743 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:48.772808 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:48.773109 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:48.773112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:48.773115 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:48.773203 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.773206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.774252 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.774253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.775641 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:48.775644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:48.776696 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:48.776697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:48.776769 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:48.776774 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:48.777644 (+ 870us) client_negotiation.cc:240] Negotiation successful
0807 08:37:48.777716 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":74,"threads_started":1}
I20260807 08:37:48.777758 22469 tablet_service.cc:1467] Processing CreateTablet for tablet 3c314014fa2649a48aaa9d883eeb3327 (DEFAULT_TABLE table=test-table [id=d4563e7a3d774f46ace0df81c42397cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:48.778179 22469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c314014fa2649a48aaa9d883eeb3327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:48.778509 22252 tablet_service.cc:1467] Processing CreateTablet for tablet 3c314014fa2649a48aaa9d883eeb3327 (DEFAULT_TABLE table=test-table [id=d4563e7a3d774f46ace0df81c42397cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:48.778626 22025 tablet_service.cc:1467] Processing CreateTablet for tablet 3c314014fa2649a48aaa9d883eeb3327 (DEFAULT_TABLE table=test-table [id=d4563e7a3d774f46ace0df81c42397cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:48.778959 22025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c314014fa2649a48aaa9d883eeb3327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:48.779040 22252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c314014fa2649a48aaa9d883eeb3327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:48.782207 22590 tablet_bootstrap.cc:492] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Bootstrap starting.
I20260807 08:37:48.783088 22590 tablet_bootstrap.cc:654] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:48.783109 22592 tablet_bootstrap.cc:492] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Bootstrap starting.
I20260807 08:37:48.783463 22590 log.cc:824] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:48.783727 22592 tablet_bootstrap.cc:654] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:48.783864 22591 tablet_bootstrap.cc:492] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Bootstrap starting.
I20260807 08:37:48.784106 22592 log.cc:824] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:48.784394 22590 tablet_bootstrap.cc:492] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: No bootstrap required, opened a new log
I20260807 08:37:48.784483 22590 ts_tablet_manager.cc:1397] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:37:48.784912 22592 tablet_bootstrap.cc:492] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: No bootstrap required, opened a new log
I20260807 08:37:48.785009 22592 ts_tablet_manager.cc:1397] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:48.786302 22590 raft_consensus.cc:348] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:48.786453 22590 raft_consensus.cc:374] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:48.786476 22590 raft_consensus.cc:729] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181852bc61e24d4387a01f2b9527663a, State: Initialized, Role: FOLLOWER
I20260807 08:37:48.786643 22590 consensus_queue.cc:260] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:48.786965 22590 ts_tablet_manager.cc:1428] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:37:48.786890 22592 raft_consensus.cc:348] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:48.787035 22551 heartbeater.cc:503] Master 127.10.207.62:42723 was elected leader, sending a full tablet report...
I20260807 08:37:48.787091 22592 raft_consensus.cc:374] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:48.787137 22592 raft_consensus.cc:729] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Initialized, Role: FOLLOWER
I20260807 08:37:48.787279 22592 consensus_queue.cc:260] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:48.787587 22592 ts_tablet_manager.cc:1428] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:37:48.787632 22136 heartbeater.cc:503] Master 127.10.207.62:42723 was elected leader, sending a full tablet report...
I20260807 08:37:48.787693 22591 tablet_bootstrap.cc:654] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:48.788028 22591 log.cc:824] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:48.789131 22591 tablet_bootstrap.cc:492] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: No bootstrap required, opened a new log
I20260807 08:37:48.789233 22591 ts_tablet_manager.cc:1397] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 08:37:48.791065 22591 raft_consensus.cc:348] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:48.791199 22591 raft_consensus.cc:374] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:48.791224 22591 raft_consensus.cc:729] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213d0af2d18649d0afaf6b08f283d9c9, State: Initialized, Role: FOLLOWER
I20260807 08:37:48.791360 22591 consensus_queue.cc:260] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:48.791627 22591 ts_tablet_manager.cc:1428] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:37:48.791726 22360 heartbeater.cc:503] Master 127.10.207.62:42723 was elected leader, sending a full tablet report...
W20260807 08:37:48.867607 22137 tablet.cc:2369] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:48.969275 22552 tablet.cc:2369] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:37:49.037611 22361 tablet.cc:2369] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:37:49.152642 22600 raft_consensus.cc:482] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:49.152856 22600 raft_consensus.cc:504] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:49.153398 22600 leader_election.cc:288] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141), 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709)
I20260807 08:37:49.157305 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.153782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:33413 (local address 127.10.207.3:39141)
0807 08:37:49.153924 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.153927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.154033 (+ 106us) server_negotiation.cc:407] Connection header received
0807 08:37:49.154157 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.154158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.154171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.154205 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:49.154464 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.155193 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.156440 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.156939 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.157020 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.157030 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.157194 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.157230 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 08:37:49.157330 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.153624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:39141 (local address 127.10.207.2:33413)
0807 08:37:49.153950 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.153963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:49.154072 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:49.154291 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:49.154292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:49.154295 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:49.154379 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.154381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.155303 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.155304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.156302 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.156305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.157032 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.157033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.157116 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.157120 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:49.157187 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:37:49.157238 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":64,"threads_started":1}
I20260807 08:37:49.157713 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c314014fa2649a48aaa9d883eeb3327" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a" is_pre_election: true
I20260807 08:37:49.157917 22491 raft_consensus.cc:2454] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 0.
I20260807 08:37:49.158149 22176 leader_election.cc:302] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181852bc61e24d4387a01f2b9527663a, 213d0af2d18649d0afaf6b08f283d9c9; no voters:
I20260807 08:37:49.158303 22600 raft_consensus.cc:2781] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:49.158367 22600 raft_consensus.cc:482] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:49.158392 22600 raft_consensus.cc:3037] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.158501 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.154278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:41245 (local address 127.10.207.1:42709)
0807 08:37:49.154375 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.154378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.154389 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:49.154428 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.154430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.154442 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.154502 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:49.154818 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.155531 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.157309 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.158056 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.158116 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.158129 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.158348 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.158429 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:37:49.158761 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.153794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42709 (local address 127.10.207.2:41245)
0807 08:37:49.153997 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.154035 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:49.154166 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:49.154600 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:49.154602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:49.154605 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:49.154715 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.154718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.155628 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.155629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.157182 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.157186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.158156 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.158157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.158242 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.158246 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:49.158642 (+ 396us) client_negotiation.cc:240] Negotiation successful
0807 08:37:49.158694 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":38,"threads_started":1}
I20260807 08:37:49.159150 22054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c314014fa2649a48aaa9d883eeb3327" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0cec708264197bd993cbed085f442" is_pre_election: true
I20260807 08:37:49.159358 22054 raft_consensus.cc:2454] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 0.
I20260807 08:37:49.159411 22600 raft_consensus.cc:504] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:49.159559 22600 leader_election.cc:288] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 election: Requested vote from peers 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141), 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709)
I20260807 08:37:49.159758 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c314014fa2649a48aaa9d883eeb3327" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a"
I20260807 08:37:49.159852 22491 raft_consensus.cc:3037] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.159896 22054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c314014fa2649a48aaa9d883eeb3327" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0cec708264197bd993cbed085f442"
I20260807 08:37:49.159997 22054 raft_consensus.cc:3037] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.160830 22491 raft_consensus.cc:2454] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 1.
I20260807 08:37:49.161044 22176 leader_election.cc:302] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181852bc61e24d4387a01f2b9527663a, 213d0af2d18649d0afaf6b08f283d9c9; no voters:
I20260807 08:37:49.161149 22600 raft_consensus.cc:2781] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:49.161250 22054 raft_consensus.cc:2454] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 1.
I20260807 08:37:49.161275 22600 raft_consensus.cc:686] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 LEADER]: Becoming Leader. State: Replica: 213d0af2d18649d0afaf6b08f283d9c9, State: Running, Role: LEADER
I20260807 08:37:49.161376 22600 consensus_queue.cc:237] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } }
I20260807 08:37:49.162425 21804 catalog_manager.cc:5495] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2). New cstate: current_term: 1 leader_uuid: "213d0af2d18649d0afaf6b08f283d9c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:49.221165 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.218450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.10.207.2:38545)
0807 08:37:49.218600 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.218603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.218612 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:49.218668 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.218671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.218700 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.218792 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:49.218948 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.219475 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.220285 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.220503 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.220597 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.220787 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:49.220921 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:49.221018 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.221042 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.221066 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:37:49.223688 22491 raft_consensus.cc:1261] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Refusing update from remote peer 213d0af2d18649d0afaf6b08f283d9c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.223717 22054 raft_consensus.cc:1261] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Refusing update from remote peer 213d0af2d18649d0afaf6b08f283d9c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.224112 22600 consensus_queue.cc:1035] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.224388 22603 consensus_queue.cc:1035] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.227926 22608 mvcc.cc:205] Tried to move back new op lower bound from 7315832296336834560 to 7315832296085532672. Current Snapshot: MvccSnapshot[applied={T|T < 7315832296336834560}]
I20260807 08:37:49.230967 21804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:49.231065 21804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:49.232048 21804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Sending DeleteTablet for 3 replicas of tablet 3c314014fa2649a48aaa9d883eeb3327
I20260807 08:37:49.232470 22252 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c314014fa2649a48aaa9d883eeb3327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46746
I20260807 08:37:49.232565 22469 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c314014fa2649a48aaa9d883eeb3327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53188
I20260807 08:37:49.232631 22025 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c314014fa2649a48aaa9d883eeb3327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53738
I20260807 08:37:49.232873 22621 tablet_replica.cc:330] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: stopping tablet replica
I20260807 08:37:49.232998 22621 raft_consensus.cc:2229] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:49.232949 21804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:49.233098 21804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:49.233124 22621 raft_consensus.cc:2258] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.233332 22623 tablet_replica.cc:330] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: stopping tablet replica
I20260807 08:37:49.233436 22623 raft_consensus.cc:2229] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:49.233539 22623 raft_consensus.cc:2258] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.233705 22624 tablet_replica.cc:330] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: stopping tablet replica
I20260807 08:37:49.233827 22624 raft_consensus.cc:2229] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:49.234009 22621 ts_tablet_manager.cc:1905] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.234109 22624 raft_consensus.cc:2258] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.234705 22623 ts_tablet_manager.cc:1905] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.235327 22025 tablet_service.cc:1467] Processing CreateTablet for tablet 113ced464a564123a545f809e497f4e8 (DEFAULT_TABLE table=test-table [id=8c8df468266747d2b5bf3d129449bdd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.235477 22025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 113ced464a564123a545f809e497f4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.236336 22621 ts_tablet_manager.cc:1918] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:49.236487 22621 log.cc:1196] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/3c314014fa2649a48aaa9d883eeb3327
I20260807 08:37:49.236562 22623 ts_tablet_manager.cc:1918] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:49.236657 22623 log.cc:1196] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/3c314014fa2649a48aaa9d883eeb3327
I20260807 08:37:49.236960 22623 ts_tablet_manager.cc:1939] T 3c314014fa2649a48aaa9d883eeb3327 P 181852bc61e24d4387a01f2b9527663a: Deleting consensus metadata
I20260807 08:37:49.237154 22621 ts_tablet_manager.cc:1939] T 3c314014fa2649a48aaa9d883eeb3327 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting consensus metadata
I20260807 08:37:49.237210 22469 tablet_service.cc:1467] Processing CreateTablet for tablet 113ced464a564123a545f809e497f4e8 (DEFAULT_TABLE table=test-table [id=8c8df468266747d2b5bf3d129449bdd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.237329 22469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 113ced464a564123a545f809e497f4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.237742 21782 catalog_manager.cc:4841] TS 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545): tablet 3c314014fa2649a48aaa9d883eeb3327 (table test-table [id=d4563e7a3d774f46ace0df81c42397cc]) successfully deleted
I20260807 08:37:49.238523 22590 tablet_bootstrap.cc:492] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: Bootstrap starting.
I20260807 08:37:49.238596 22252 tablet_service.cc:1467] Processing CreateTablet for tablet 113ced464a564123a545f809e497f4e8 (DEFAULT_TABLE table=test-table [id=8c8df468266747d2b5bf3d129449bdd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.238713 22252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 113ced464a564123a545f809e497f4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.238948 22590 tablet_bootstrap.cc:654] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.240106 22591 tablet_bootstrap.cc:492] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: Bootstrap starting.
I20260807 08:37:49.240242 22592 tablet_bootstrap.cc:492] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: Bootstrap starting.
I20260807 08:37:49.240603 22592 tablet_bootstrap.cc:654] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.240598 22591 tablet_bootstrap.cc:654] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.241480 21784 catalog_manager.cc:4841] TS 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141): tablet 3c314014fa2649a48aaa9d883eeb3327 (table test-table [id=d4563e7a3d774f46ace0df81c42397cc]) successfully deleted
I20260807 08:37:49.241618 22591 tablet_bootstrap.cc:492] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: No bootstrap required, opened a new log
I20260807 08:37:49.241678 22591 ts_tablet_manager.cc:1397] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:49.241842 22591 raft_consensus.cc:348] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.241932 22591 raft_consensus.cc:374] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.241958 22591 raft_consensus.cc:729] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213d0af2d18649d0afaf6b08f283d9c9, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.242017 22591 consensus_queue.cc:260] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.242169 22591 ts_tablet_manager.cc:1428] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:49.242153 22624 ts_tablet_manager.cc:1905] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.243076 22592 tablet_bootstrap.cc:492] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: No bootstrap required, opened a new log
I20260807 08:37:49.243134 22592 ts_tablet_manager.cc:1397] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 08:37:49.243280 22590 tablet_bootstrap.cc:492] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: No bootstrap required, opened a new log
I20260807 08:37:49.243331 22590 ts_tablet_manager.cc:1397] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 08:37:49.243296 22592 raft_consensus.cc:348] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.243362 22592 raft_consensus.cc:374] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.243386 22592 raft_consensus.cc:729] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.243425 22592 consensus_queue.cc:260] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.243538 22592 ts_tablet_manager.cc:1428] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:49.243520 22590 raft_consensus.cc:348] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.243597 22590 raft_consensus.cc:374] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.243619 22590 raft_consensus.cc:729] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181852bc61e24d4387a01f2b9527663a, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.243664 22590 consensus_queue.cc:260] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.243762 22590 ts_tablet_manager.cc:1428] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:49.250522 22624 ts_tablet_manager.cc:1918] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:49.250648 22624 log.cc:1196] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/3c314014fa2649a48aaa9d883eeb3327
I20260807 08:37:49.251148 22624 ts_tablet_manager.cc:1939] T 3c314014fa2649a48aaa9d883eeb3327 P 7aa0cec708264197bd993cbed085f442: Deleting consensus metadata
I20260807 08:37:49.251580 21784 catalog_manager.cc:4841] TS 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709): tablet 3c314014fa2649a48aaa9d883eeb3327 (table test-table [id=d4563e7a3d774f46ace0df81c42397cc]) successfully deleted
I20260807 08:37:49.409219 22596 raft_consensus.cc:482] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:49.409327 22596 raft_consensus.cc:504] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.409642 22596 leader_election.cc:288] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:49.413844 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.410075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:51275 (local address 127.10.207.2:38545)
0807 08:37:49.410127 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.410129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.410297 (+ 168us) server_negotiation.cc:407] Connection header received
0807 08:37:49.410389 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.410390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.410403 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.410473 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:49.410695 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.411561 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.413035 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.413418 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.413519 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.413530 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.413724 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.413761 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:37:49.413874 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.410077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:60855 (local address 127.10.207.3:39141)
0807 08:37:49.410125 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.410128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.410365 (+ 237us) server_negotiation.cc:407] Connection header received
0807 08:37:49.410437 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.410439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.410456 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.410522 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:37:49.410772 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.411561 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.413092 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.413461 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.413550 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.413561 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.413742 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.413780 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:37:49.413909 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.410018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:39141 (local address 127.10.207.1:60855)
0807 08:37:49.410252 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.410265 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:49.410368 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:49.410579 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:49.410580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:49.410583 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:49.410695 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.410698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.411703 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.411704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.412891 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.412894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.413549 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.413550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.413644 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.413648 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:37:49.413714 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:37:49.413778 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":190,"spinlock_wait_cycles":896,"thread_start_us":148,"threads_started":1}
I20260807 08:37:49.414111 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.410018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:38545 (local address 127.10.207.1:51275)
0807 08:37:49.410202 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.410215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:49.410317 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:49.410480 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:49.410481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:49.410484 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:37:49.410616 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.410619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.411732 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.411733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.412893 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.412898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.413537 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.413538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.413630 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.413635 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:37:49.413703 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:37:49.413761 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":64,"thread_start_us":109,"threads_started":1}
I20260807 08:37:49.414196 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113ced464a564123a545f809e497f4e8" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a" is_pre_election: true
I20260807 08:37:49.414369 22491 raft_consensus.cc:2454] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 0.
I20260807 08:37:49.414533 22290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113ced464a564123a545f809e497f4e8" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213d0af2d18649d0afaf6b08f283d9c9" is_pre_election: true
I20260807 08:37:49.414660 21951 leader_election.cc:302] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181852bc61e24d4387a01f2b9527663a, 7aa0cec708264197bd993cbed085f442; no voters:
I20260807 08:37:49.414732 22290 raft_consensus.cc:2454] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 0.
I20260807 08:37:49.414821 22596 raft_consensus.cc:2781] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:49.414873 22596 raft_consensus.cc:482] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:49.414914 22596 raft_consensus.cc:3037] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.415735 22596 raft_consensus.cc:504] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.415894 22596 leader_election.cc:288] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 election: Requested vote from peers 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:49.416086 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113ced464a564123a545f809e497f4e8" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a"
I20260807 08:37:49.416086 22290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "113ced464a564123a545f809e497f4e8" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213d0af2d18649d0afaf6b08f283d9c9"
I20260807 08:37:49.416177 22491 raft_consensus.cc:3037] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.416177 22290 raft_consensus.cc:3037] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.417183 22491 raft_consensus.cc:2454] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 1.
I20260807 08:37:49.417228 22290 raft_consensus.cc:2454] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 1.
I20260807 08:37:49.417426 21948 leader_election.cc:302] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213d0af2d18649d0afaf6b08f283d9c9, 7aa0cec708264197bd993cbed085f442; no voters:
I20260807 08:37:49.417567 22596 raft_consensus.cc:2781] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:49.417752 22596 raft_consensus.cc:686] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Running, Role: LEADER
I20260807 08:37:49.417835 22596 consensus_queue.cc:237] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.418731 21804 catalog_manager.cc:5495] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa0cec708264197bd993cbed085f442 (127.10.207.1). New cstate: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:37:49 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091868, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.62@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:49.450874 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.439641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:42723 (local address 127.0.0.1:55758)
0807 08:37:49.439846 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.439859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:49.439946 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:49.440118 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:49.440119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:49.440566 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:49.440636 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.440638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.441392 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.441393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.441997 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.441999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.442382 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.442383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.442432 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.445902 (+ 3470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:49.445908 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:49.447620 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:49.449500 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:49.449508 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:49.449511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:49.449754 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:49.450016 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:49.450017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:49.450017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:49.450087 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:49.450444 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:49.450452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:49.450510 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 08:37:49.450655 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:37:49.450783 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":120,"threads_started":1}
I20260807 08:37:49.451066 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.439759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.10.207.62:42723)
0807 08:37:49.439919 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.439921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.439932 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:37:49.440012 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.440013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.440024 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.440083 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:49.440761 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.441277 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.442095 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.442295 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.442363 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.447733 (+ 5370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:49.447735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:49.447735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:49.447744 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:49.449397 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:49.449864 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:49.449866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:49.449866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:49.449940 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:49.450190 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:49.450191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:49.450191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:49.450341 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:49.450412 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.450729 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.450866 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260807 08:37:49.453660 21804 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:55758:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:37:49.453918 21804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:49.456068 22252 tablet_service.cc:1467] Processing CreateTablet for tablet 0005ee46c2ed425a8b8888ae8fa52dbe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2466bf67770c456ead8657fa16a60b8a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:49.456156 22469 tablet_service.cc:1467] Processing CreateTablet for tablet 0005ee46c2ed425a8b8888ae8fa52dbe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2466bf67770c456ead8657fa16a60b8a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:49.456167 22025 tablet_service.cc:1467] Processing CreateTablet for tablet 0005ee46c2ed425a8b8888ae8fa52dbe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2466bf67770c456ead8657fa16a60b8a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:37:49.456326 22252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0005ee46c2ed425a8b8888ae8fa52dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.456341 22025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0005ee46c2ed425a8b8888ae8fa52dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.456341 22469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0005ee46c2ed425a8b8888ae8fa52dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.457916 22591 tablet_bootstrap.cc:492] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: Bootstrap starting.
I20260807 08:37:49.458016 22592 tablet_bootstrap.cc:492] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: Bootstrap starting.
I20260807 08:37:49.458026 22590 tablet_bootstrap.cc:492] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: Bootstrap starting.
I20260807 08:37:49.458335 22591 tablet_bootstrap.cc:654] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.458513 22590 tablet_bootstrap.cc:654] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.458606 22592 tablet_bootstrap.cc:654] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.459182 22591 tablet_bootstrap.cc:492] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: No bootstrap required, opened a new log
I20260807 08:37:49.459239 22591 ts_tablet_manager.cc:1397] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:49.459408 22591 raft_consensus.cc:348] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.459501 22591 raft_consensus.cc:374] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.459529 22591 raft_consensus.cc:729] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213d0af2d18649d0afaf6b08f283d9c9, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.459604 22592 tablet_bootstrap.cc:492] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: No bootstrap required, opened a new log
I20260807 08:37:49.459584 22591 consensus_queue.cc:260] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.459645 22592 ts_tablet_manager.cc:1397] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:37:49.459719 22591 ts_tablet_manager.cc:1428] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:49.459817 22592 raft_consensus.cc:348] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.459766 22600 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.459905 22592 raft_consensus.cc:374] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.459924 22592 raft_consensus.cc:729] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.459914 22600 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:49.459981 22592 consensus_queue.cc:260] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.460115 22592 ts_tablet_manager.cc:1428] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 08:37:49.460290 22596 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.460376 22596 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:49.460505 22590 tablet_bootstrap.cc:492] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: No bootstrap required, opened a new log
I20260807 08:37:49.460565 22590 ts_tablet_manager.cc:1397] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 08:37:49.460745 22590 raft_consensus.cc:348] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.460840 22590 raft_consensus.cc:374] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.460870 22590 raft_consensus.cc:729] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181852bc61e24d4387a01f2b9527663a, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.460933 22590 consensus_queue.cc:260] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.461041 22590 ts_tablet_manager.cc:1428] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:49.461135 22595 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.461264 22595 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:49.649070 22596 raft_consensus.cc:482] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:49.649189 22596 raft_consensus.cc:504] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.649451 22596 leader_election.cc:288] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:49.649816 22290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0005ee46c2ed425a8b8888ae8fa52dbe" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213d0af2d18649d0afaf6b08f283d9c9" is_pre_election: true
I20260807 08:37:49.649816 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0005ee46c2ed425a8b8888ae8fa52dbe" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a" is_pre_election: true
I20260807 08:37:49.649986 22491 raft_consensus.cc:2454] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 0.
I20260807 08:37:49.649986 22290 raft_consensus.cc:2454] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 0.
I20260807 08:37:49.650213 21951 leader_election.cc:302] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181852bc61e24d4387a01f2b9527663a, 7aa0cec708264197bd993cbed085f442; no voters:
I20260807 08:37:49.650368 22596 raft_consensus.cc:2781] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:49.650439 22596 raft_consensus.cc:482] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:49.650454 22596 raft_consensus.cc:3037] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.651188 22596 raft_consensus.cc:504] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.651335 22596 leader_election.cc:288] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 election: Requested vote from peers 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:49.651526 22290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0005ee46c2ed425a8b8888ae8fa52dbe" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213d0af2d18649d0afaf6b08f283d9c9"
I20260807 08:37:49.651523 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0005ee46c2ed425a8b8888ae8fa52dbe" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a"
I20260807 08:37:49.651616 22290 raft_consensus.cc:3037] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.651638 22491 raft_consensus.cc:3037] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.652261 22290 raft_consensus.cc:2454] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 1.
I20260807 08:37:49.652473 21948 leader_election.cc:302] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213d0af2d18649d0afaf6b08f283d9c9, 7aa0cec708264197bd993cbed085f442; no voters:
I20260807 08:37:49.652581 22596 raft_consensus.cc:2781] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:49.652653 22491 raft_consensus.cc:2454] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 1.
I20260807 08:37:49.652666 22596 raft_consensus.cc:686] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Running, Role: LEADER
I20260807 08:37:49.652736 22596 consensus_queue.cc:237] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.653101 22631 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: TxnStatusTablet state changed. Reason: New leader 7aa0cec708264197bd993cbed085f442. Latest consensus state: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.653196 22631 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:49.653419 22649 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:37:49.653553 22649 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:37:49.653512 21804 catalog_manager.cc:5495] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa0cec708264197bd993cbed085f442 (127.10.207.1). New cstate: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:49.690665 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.687888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42709 (local address 127.0.0.1:41824)
0807 08:37:49.687979 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.687992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:49.688055 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:49.688353 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:49.688354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:49.688355 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:49.688410 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.688413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.689094 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.689095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.689674 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.689676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.690086 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.690087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.690145 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.690161 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:49.690462 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:49.690473 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 08:37:49.690518 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:37:49.690592 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 08:37:49.690682 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.687995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.10.207.1:42709)
0807 08:37:49.688200 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.688203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.688209 (+ 6us) server_negotiation.cc:407] Connection header received
0807 08:37:49.688240 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.688240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.688259 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.688345 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:49.688469 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.689006 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.689764 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.690005 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.690083 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.690277 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:49.690386 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:49.690476 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.690563 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.690587 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
I20260807 08:37:49.691890 22491 raft_consensus.cc:1261] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Refusing update from remote peer 7aa0cec708264197bd993cbed085f442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.691982 22290 raft_consensus.cc:1261] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Refusing update from remote peer 7aa0cec708264197bd993cbed085f442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.692152 22596 consensus_queue.cc:1035] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.692342 22596 consensus_queue.cc:1035] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.693401 22600 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: TxnStatusTablet state changed. Reason: New leader 7aa0cec708264197bd993cbed085f442. Latest consensus state: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.693493 22600 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:49.693676 22595 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: TxnStatusTablet state changed. Reason: New leader 7aa0cec708264197bd993cbed085f442. Latest consensus state: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.693820 22595 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:49.694300 22631 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.694399 22631 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:49.694496 22607 mvcc.cc:205] Tried to move back new op lower bound from 7315832298255970304 to 7315832298098016256. Current Snapshot: MvccSnapshot[applied={T|T < 7315832298255970304}]
I20260807 08:37:49.694597 22600 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.694669 22600 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 213d0af2d18649d0afaf6b08f283d9c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:49.695731 22596 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.695827 22596 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 7aa0cec708264197bd993cbed085f442: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:37:49.696132 22609 mvcc.cc:205] Tried to move back new op lower bound from 7315832298255970304 to 7315832298098016256. Current Snapshot: MvccSnapshot[applied={T|T < 7315832298255970304}]
I20260807 08:37:49.696244 22595 tablet_replica.cc:439] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } } }
I20260807 08:37:49.696354 22595 tablet_replica.cc:442] T 0005ee46c2ed425a8b8888ae8fa52dbe P 181852bc61e24d4387a01f2b9527663a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:37:49.699285 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.696410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.10.207.1:42709)
0807 08:37:49.696487 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.696489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.696497 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:49.696515 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.696516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.696537 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.696851 (+ 314us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:49.696881 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.697414 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.698631 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.698827 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.698896 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.699074 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:49.699098 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:49.699144 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.699217 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.699232 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:37:49.704879 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.702013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:41051 (local address 127.10.207.1:42709)
0807 08:37:49.702085 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.702087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.702257 (+ 170us) server_negotiation.cc:407] Connection header received
0807 08:37:49.702314 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.702314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.702324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.702373 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:49.702589 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.703005 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.704166 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.704350 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.704407 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.704633 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:49.704655 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:49.704697 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.704790 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.704834 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":40}
I20260807 08:37:49.704996 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.701914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42709 (local address 127.10.207.1:41051)
0807 08:37:49.702169 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:49.702179 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:49.702260 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:49.702397 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:49.702398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:49.702399 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:49.702481 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.702483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.703091 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.703092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.704074 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:49.704077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.704432 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.704434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:49.704519 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.704536 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:49.704723 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:49.704725 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:49.704786 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:37:49.704864 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":6,"thread_start_us":114,"threads_started":1}
I20260807 08:37:49.707348 22290 raft_consensus.cc:1261] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Refusing update from remote peer 7aa0cec708264197bd993cbed085f442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.707405 22491 raft_consensus.cc:1261] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Refusing update from remote peer 7aa0cec708264197bd993cbed085f442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.707561 22658 consensus_queue.cc:1035] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.707635 22631 consensus_queue.cc:1035] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.722913 21805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:49.723040 21805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:49.723882 21805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Sending DeleteTablet for 3 replicas of tablet 113ced464a564123a545f809e497f4e8
I20260807 08:37:49.724179 22252 tablet_service.cc:1514] Processing DeleteTablet for tablet 113ced464a564123a545f809e497f4e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46746
I20260807 08:37:49.724294 22621 tablet_replica.cc:330] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: stopping tablet replica
I20260807 08:37:49.724360 22621 raft_consensus.cc:2229] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:49.724402 22621 raft_consensus.cc:2258] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.724761 22469 tablet_service.cc:1514] Processing DeleteTablet for tablet 113ced464a564123a545f809e497f4e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53188
I20260807 08:37:49.724881 22623 tablet_replica.cc:330] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: stopping tablet replica
I20260807 08:37:49.724932 22623 raft_consensus.cc:2229] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:49.724973 22623 raft_consensus.cc:2258] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.724982 22621 ts_tablet_manager.cc:1905] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.727015 22621 ts_tablet_manager.cc:1918] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:49.727093 22621 log.cc:1196] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/113ced464a564123a545f809e497f4e8
I20260807 08:37:49.727346 22621 ts_tablet_manager.cc:1939] T 113ced464a564123a545f809e497f4e8 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting consensus metadata
I20260807 08:37:49.727916 21782 catalog_manager.cc:4841] TS 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545): tablet 113ced464a564123a545f809e497f4e8 (table test-table [id=8c8df468266747d2b5bf3d129449bdd6]) successfully deleted
I20260807 08:37:49.728097 22025 tablet_service.cc:1514] Processing DeleteTablet for tablet 113ced464a564123a545f809e497f4e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53738
I20260807 08:37:49.728214 22624 tablet_replica.cc:330] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: stopping tablet replica
I20260807 08:37:49.728271 22624 raft_consensus.cc:2229] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:49.728382 22624 raft_consensus.cc:2258] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.728698 22623 ts_tablet_manager.cc:1905] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.730711 22623 ts_tablet_manager.cc:1918] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:49.730793 22623 log.cc:1196] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/113ced464a564123a545f809e497f4e8
I20260807 08:37:49.731032 22623 ts_tablet_manager.cc:1939] T 113ced464a564123a545f809e497f4e8 P 181852bc61e24d4387a01f2b9527663a: Deleting consensus metadata
I20260807 08:37:49.731740 22624 ts_tablet_manager.cc:1905] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.733570 22624 ts_tablet_manager.cc:1918] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:49.733642 22624 log.cc:1196] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/113ced464a564123a545f809e497f4e8
I20260807 08:37:49.733881 22624 ts_tablet_manager.cc:1939] T 113ced464a564123a545f809e497f4e8 P 7aa0cec708264197bd993cbed085f442: Deleting consensus metadata
I20260807 08:37:49.734190 21784 catalog_manager.cc:4841] TS 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141): tablet 113ced464a564123a545f809e497f4e8 (table test-table [id=8c8df468266747d2b5bf3d129449bdd6]) successfully deleted
I20260807 08:37:49.734292 21784 catalog_manager.cc:4841] TS 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709): tablet 113ced464a564123a545f809e497f4e8 (table test-table [id=8c8df468266747d2b5bf3d129449bdd6]) successfully deleted
Aug 07 08:37:49 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:37:49 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:49 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:37:49.752161 22641 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:49.744662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.10.207.62:42723)
0807 08:37:49.744740 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.744745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.744891 (+ 146us) server_negotiation.cc:407] Connection header received
0807 08:37:49.744948 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.744949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.744965 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.745059 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:49.745669 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.746505 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.747636 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.747958 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.748064 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.752083 (+ 4019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55764: BlockingRecv error: recv got EOF from 127.0.0.1:55764 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 08:37:49 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM
I20260807 08:37:49.760040 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.752936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.10.207.62:42723)
0807 08:37:49.752996 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.752999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.753009 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:49.753071 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.753072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.753083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.753139 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:49.753603 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.754098 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.754845 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.755042 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.755116 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.757227 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:49.757228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:49.757229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:49.757236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:49.758873 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:49.759248 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:49.759250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:49.759250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:49.759308 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:49.759525 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:49.759526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:49.759526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:49.759633 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:49.759744 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.759847 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.759934 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:37:49.761759 21805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:49.762022 21805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:49.763711 22252 tablet_service.cc:1467] Processing CreateTablet for tablet bceda61cf4c14337bf3d7f792b9a02fd (DEFAULT_TABLE table=test-table [id=7a3db3306ecb42c4a8cdd9225982d159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.763883 22252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bceda61cf4c14337bf3d7f792b9a02fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.763867 22025 tablet_service.cc:1467] Processing CreateTablet for tablet bceda61cf4c14337bf3d7f792b9a02fd (DEFAULT_TABLE table=test-table [id=7a3db3306ecb42c4a8cdd9225982d159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.764047 22025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bceda61cf4c14337bf3d7f792b9a02fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.765760 22592 tablet_bootstrap.cc:492] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: Bootstrap starting.
I20260807 08:37:49.765821 22469 tablet_service.cc:1467] Processing CreateTablet for tablet bceda61cf4c14337bf3d7f792b9a02fd (DEFAULT_TABLE table=test-table [id=7a3db3306ecb42c4a8cdd9225982d159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.765985 22469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bceda61cf4c14337bf3d7f792b9a02fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.766275 22592 tablet_bootstrap.cc:654] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.767016 22592 tablet_bootstrap.cc:492] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: No bootstrap required, opened a new log
I20260807 08:37:49.767082 22592 ts_tablet_manager.cc:1397] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:49.767217 22592 raft_consensus.cc:348] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.767303 22592 raft_consensus.cc:374] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.767341 22592 raft_consensus.cc:729] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.767383 22592 consensus_queue.cc:260] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.767491 22592 ts_tablet_manager.cc:1428] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:49.767699 22590 tablet_bootstrap.cc:492] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: Bootstrap starting.
I20260807 08:37:49.767741 22591 tablet_bootstrap.cc:492] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: Bootstrap starting.
I20260807 08:37:49.768033 22590 tablet_bootstrap.cc:654] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.768244 22591 tablet_bootstrap.cc:654] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.768720 22590 tablet_bootstrap.cc:492] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: No bootstrap required, opened a new log
I20260807 08:37:49.768775 22590 ts_tablet_manager.cc:1397] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:37:49.768965 22590 raft_consensus.cc:348] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.769050 22590 raft_consensus.cc:374] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.769073 22590 raft_consensus.cc:729] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181852bc61e24d4387a01f2b9527663a, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.769127 22590 consensus_queue.cc:260] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.769222 22590 ts_tablet_manager.cc:1428] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:49.769243 22591 tablet_bootstrap.cc:492] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: No bootstrap required, opened a new log
I20260807 08:37:49.769312 22591 ts_tablet_manager.cc:1397] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:49.769459 22591 raft_consensus.cc:348] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.769547 22591 raft_consensus.cc:374] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.769568 22591 raft_consensus.cc:729] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213d0af2d18649d0afaf6b08f283d9c9, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.769618 22591 consensus_queue.cc:260] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.769706 22591 ts_tablet_manager.cc:1428] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:49.865073 22596 raft_consensus.cc:482] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:49.865221 22596 raft_consensus.cc:504] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.865481 22596 leader_election.cc:288] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:49.865720 22290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceda61cf4c14337bf3d7f792b9a02fd" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213d0af2d18649d0afaf6b08f283d9c9" is_pre_election: true
I20260807 08:37:49.865873 22290 raft_consensus.cc:2454] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 0.
I20260807 08:37:49.865799 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceda61cf4c14337bf3d7f792b9a02fd" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a" is_pre_election: true
I20260807 08:37:49.865964 22491 raft_consensus.cc:2454] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 0.
I20260807 08:37:49.866079 21948 leader_election.cc:302] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213d0af2d18649d0afaf6b08f283d9c9, 7aa0cec708264197bd993cbed085f442; no voters:
I20260807 08:37:49.866235 22596 raft_consensus.cc:2781] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:49.866287 22596 raft_consensus.cc:482] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:49.866318 22596 raft_consensus.cc:3037] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.867209 22596 raft_consensus.cc:504] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.867373 22596 leader_election.cc:288] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 election: Requested vote from peers 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:49.867588 22290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceda61cf4c14337bf3d7f792b9a02fd" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213d0af2d18649d0afaf6b08f283d9c9"
I20260807 08:37:49.867595 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bceda61cf4c14337bf3d7f792b9a02fd" candidate_uuid: "7aa0cec708264197bd993cbed085f442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a"
I20260807 08:37:49.867703 22290 raft_consensus.cc:3037] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.867833 22491 raft_consensus.cc:3037] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:49.868636 22491 raft_consensus.cc:2454] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 1.
I20260807 08:37:49.868794 22290 raft_consensus.cc:2454] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0cec708264197bd993cbed085f442 in term 1.
I20260807 08:37:49.868957 21951 leader_election.cc:302] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181852bc61e24d4387a01f2b9527663a, 7aa0cec708264197bd993cbed085f442; no voters:
I20260807 08:37:49.869077 22596 raft_consensus.cc:2781] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:49.869134 22596 raft_consensus.cc:686] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Running, Role: LEADER
I20260807 08:37:49.869185 22596 consensus_queue.cc:237] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.869905 21805 catalog_manager.cc:5495] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa0cec708264197bd993cbed085f442 (127.10.207.1). New cstate: current_term: 1 leader_uuid: "7aa0cec708264197bd993cbed085f442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:49.892458 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:49.890061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.10.207.1:42709)
0807 08:37:49.890159 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:49.890162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:49.890171 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:37:49.890199 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:49.890201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:49.890213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:49.890283 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:49.890453 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.890999 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.891747 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:49.891945 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:49.892023 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:49.892178 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:49.892218 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:49.892281 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:49.892342 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 08:37:49.892360 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:37:49.893410 22491 raft_consensus.cc:1261] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Refusing update from remote peer 7aa0cec708264197bd993cbed085f442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.893426 22290 raft_consensus.cc:1261] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Refusing update from remote peer 7aa0cec708264197bd993cbed085f442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:49.893663 22631 consensus_queue.cc:1035] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.893967 22631 consensus_queue.cc:1035] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:49.896756 21804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:49.896857 21804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:49.897517 21804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Sending DeleteTablet for 3 replicas of tablet bceda61cf4c14337bf3d7f792b9a02fd
I20260807 08:37:49.897801 22469 tablet_service.cc:1514] Processing DeleteTablet for tablet bceda61cf4c14337bf3d7f792b9a02fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53188
I20260807 08:37:49.897818 22252 tablet_service.cc:1514] Processing DeleteTablet for tablet bceda61cf4c14337bf3d7f792b9a02fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46746
I20260807 08:37:49.897904 22623 tablet_replica.cc:330] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: stopping tablet replica
I20260807 08:37:49.897871 22251 tablet_service.cc:1514] Processing DeleteTablet for tablet bceda61cf4c14337bf3d7f792b9a02fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46746
I20260807 08:37:49.897943 22621 tablet_replica.cc:330] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: stopping tablet replica
I20260807 08:37:49.897959 22623 raft_consensus.cc:2229] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:49.898012 22623 raft_consensus.cc:2258] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.898015 22621 raft_consensus.cc:2229] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:49.898065 22621 raft_consensus.cc:2258] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.898454 22621 ts_tablet_manager.cc:1905] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.898595 22025 tablet_service.cc:1514] Processing DeleteTablet for tablet bceda61cf4c14337bf3d7f792b9a02fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:49 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53738
I20260807 08:37:49.898692 22624 tablet_replica.cc:330] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: stopping tablet replica
I20260807 08:37:49.898746 22624 raft_consensus.cc:2229] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:49.898743 22623 ts_tablet_manager.cc:1905] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.898806 22624 raft_consensus.cc:2258] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:49.899199 22624 ts_tablet_manager.cc:1905] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:49.900077 22621 ts_tablet_manager.cc:1918] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:49.900141 22621 log.cc:1196] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/bceda61cf4c14337bf3d7f792b9a02fd
I20260807 08:37:49.900347 22623 ts_tablet_manager.cc:1918] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:49.900396 22623 log.cc:1196] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/bceda61cf4c14337bf3d7f792b9a02fd
I20260807 08:37:49.900399 22621 ts_tablet_manager.cc:1939] T bceda61cf4c14337bf3d7f792b9a02fd P 213d0af2d18649d0afaf6b08f283d9c9: Deleting consensus metadata
I20260807 08:37:49.900622 22623 ts_tablet_manager.cc:1939] T bceda61cf4c14337bf3d7f792b9a02fd P 181852bc61e24d4387a01f2b9527663a: Deleting consensus metadata
I20260807 08:37:49.901034 21784 catalog_manager.cc:4841] TS 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141): tablet bceda61cf4c14337bf3d7f792b9a02fd (table test-table [id=7a3db3306ecb42c4a8cdd9225982d159]) successfully deleted
I20260807 08:37:49.901065 22624 ts_tablet_manager.cc:1918] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:37:49.901135 22624 log.cc:1196] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/bceda61cf4c14337bf3d7f792b9a02fd
I20260807 08:37:49.901409 22624 ts_tablet_manager.cc:1939] T bceda61cf4c14337bf3d7f792b9a02fd P 7aa0cec708264197bd993cbed085f442: Deleting consensus metadata
I20260807 08:37:49.901785 21784 catalog_manager.cc:4841] TS 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709): tablet bceda61cf4c14337bf3d7f792b9a02fd (table test-table [id=7a3db3306ecb42c4a8cdd9225982d159]) successfully deleted
I20260807 08:37:49.901880 21782 catalog_manager.cc:4841] TS 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545): tablet bceda61cf4c14337bf3d7f792b9a02fd (table test-table [id=7a3db3306ecb42c4a8cdd9225982d159]) successfully deleted
I20260807 08:37:49.898468 21804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:37:49.901996 21782 catalog_manager.cc:4808] TS 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545): delete failed for tablet bceda61cf4c14337bf3d7f792b9a02fd because the tablet was not found. No further retry: Not found: Tablet not found: bceda61cf4c14337bf3d7f792b9a02fd
W20260807 08:37:49.902043 21804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:37:49.903461 22469 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3db7521ae04ee8a793ebcde6d27739 (DEFAULT_TABLE table=test-table [id=514f8ecdcc3241ef9790c8f1b3ec264e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.903479 22251 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3db7521ae04ee8a793ebcde6d27739 (DEFAULT_TABLE table=test-table [id=514f8ecdcc3241ef9790c8f1b3ec264e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.903481 22025 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3db7521ae04ee8a793ebcde6d27739 (DEFAULT_TABLE table=test-table [id=514f8ecdcc3241ef9790c8f1b3ec264e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:37:49.903581 22469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3db7521ae04ee8a793ebcde6d27739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.903582 22251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3db7521ae04ee8a793ebcde6d27739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.903609 22025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3db7521ae04ee8a793ebcde6d27739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:49.904803 22590 tablet_bootstrap.cc:492] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: Bootstrap starting.
I20260807 08:37:49.905074 22591 tablet_bootstrap.cc:492] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: Bootstrap starting.
I20260807 08:37:49.905474 22591 tablet_bootstrap.cc:654] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.905652 22590 tablet_bootstrap.cc:654] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.905786 22592 tablet_bootstrap.cc:492] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442: Bootstrap starting.
I20260807 08:37:49.906212 22592 tablet_bootstrap.cc:654] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:49.906339 22590 tablet_bootstrap.cc:492] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: No bootstrap required, opened a new log
I20260807 08:37:49.906401 22590 ts_tablet_manager.cc:1397] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:49.906576 22590 raft_consensus.cc:348] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.906641 22590 raft_consensus.cc:374] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.906663 22590 raft_consensus.cc:729] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181852bc61e24d4387a01f2b9527663a, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.906723 22590 consensus_queue.cc:260] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.906816 22590 ts_tablet_manager.cc:1428] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:49.906850 22591 tablet_bootstrap.cc:492] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: No bootstrap required, opened a new log
I20260807 08:37:49.906899 22591 ts_tablet_manager.cc:1397] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:37:49.907034 22591 raft_consensus.cc:348] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.907135 22591 raft_consensus.cc:374] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.907157 22591 raft_consensus.cc:729] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213d0af2d18649d0afaf6b08f283d9c9, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.907208 22591 consensus_queue.cc:260] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.907315 22591 ts_tablet_manager.cc:1428] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:37:49.907963 22592 tablet_bootstrap.cc:492] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442: No bootstrap required, opened a new log
I20260807 08:37:49.908021 22592 ts_tablet_manager.cc:1397] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 08:37:49.908205 22592 raft_consensus.cc:348] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.908288 22592 raft_consensus.cc:374] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:49.908308 22592 raft_consensus.cc:729] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0cec708264197bd993cbed085f442, State: Initialized, Role: FOLLOWER
I20260807 08:37:49.908346 22592 consensus_queue.cc:260] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:49.908447 22592 ts_tablet_manager.cc:1428] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:50.012368 22600 raft_consensus.cc:482] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:37:50.012513 22600 raft_consensus.cc:504] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:50.012744 22600 leader_election.cc:288] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:50.013170 22054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3db7521ae04ee8a793ebcde6d27739" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0cec708264197bd993cbed085f442" is_pre_election: true
I20260807 08:37:50.013216 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3db7521ae04ee8a793ebcde6d27739" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a" is_pre_election: true
I20260807 08:37:50.013332 22054 raft_consensus.cc:2454] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 0.
I20260807 08:37:50.013324 22491 raft_consensus.cc:2454] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 0.
I20260807 08:37:50.013618 22176 leader_election.cc:302] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213d0af2d18649d0afaf6b08f283d9c9, 7aa0cec708264197bd993cbed085f442; no voters:
I20260807 08:37:50.013794 22600 raft_consensus.cc:2781] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:37:50.013875 22600 raft_consensus.cc:482] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:37:50.013892 22600 raft_consensus.cc:3037] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:50.014820 22600 raft_consensus.cc:504] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:50.014938 22600 leader_election.cc:288] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 election: Requested vote from peers 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709), 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141)
I20260807 08:37:50.015141 22054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3db7521ae04ee8a793ebcde6d27739" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0cec708264197bd993cbed085f442"
I20260807 08:37:50.015242 22054 raft_consensus.cc:3037] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:50.015242 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3db7521ae04ee8a793ebcde6d27739" candidate_uuid: "213d0af2d18649d0afaf6b08f283d9c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181852bc61e24d4387a01f2b9527663a"
I20260807 08:37:50.015344 22491 raft_consensus.cc:3037] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:50.016072 22491 raft_consensus.cc:2454] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 1.
I20260807 08:37:50.016530 22054 raft_consensus.cc:2454] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213d0af2d18649d0afaf6b08f283d9c9 in term 1.
I20260807 08:37:50.016692 22176 leader_election.cc:302] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181852bc61e24d4387a01f2b9527663a, 213d0af2d18649d0afaf6b08f283d9c9; no voters:
I20260807 08:37:50.016839 22600 raft_consensus.cc:2781] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:50.016990 22600 raft_consensus.cc:686] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 LEADER]: Becoming Leader. State: Replica: 213d0af2d18649d0afaf6b08f283d9c9, State: Running, Role: LEADER
I20260807 08:37:50.017059 22600 consensus_queue.cc:237] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } }
I20260807 08:37:50.017792 21804 catalog_manager.cc:5495] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2). New cstate: current_term: 1 leader_uuid: "213d0af2d18649d0afaf6b08f283d9c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "213d0af2d18649d0afaf6b08f283d9c9" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 38545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 } health_report { overall_health: UNKNOWN } } }
I20260807 08:37:50.028191 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.025727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50734 (local address 127.10.207.2:38545)
0807 08:37:50.025879 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:50.025881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:50.025888 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:37:50.025906 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:50.025907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:50.025915 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:50.025984 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:50.026143 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.026705 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.027442 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.027670 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.027746 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:50.027898 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:50.027932 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:50.028018 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:50.028085 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 08:37:50.028106 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260807 08:37:50.033458 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.030841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:43017 (local address 127.10.207.1:42709)
0807 08:37:50.030921 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:50.030924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:50.031019 (+ 95us) server_negotiation.cc:407] Connection header received
0807 08:37:50.031145 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:50.031147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:50.031158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:50.031216 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:50.031388 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.031901 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.032692 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.032910 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.032992 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:50.033183 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:50.033223 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:50.033286 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:50.033348 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:37:50.033373 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:37:50.033476 22704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.030743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42709 (local address 127.10.207.2:43017)
0807 08:37:50.030939 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:50.030957 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:50.031039 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:50.031221 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:50.031222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:50.031224 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:50.031289 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:50.031290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.031996 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.031997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:50.032602 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:50.032603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.032994 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.032995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:50.033057 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:50.033073 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:50.033288 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:50.033290 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:50.033343 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 08:37:50.033402 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260807 08:37:50.035709 22491 raft_consensus.cc:1261] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Refusing update from remote peer 213d0af2d18649d0afaf6b08f283d9c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:50.035794 22054 raft_consensus.cc:1261] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Refusing update from remote peer 213d0af2d18649d0afaf6b08f283d9c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:37:50.036042 22694 consensus_queue.cc:1035] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa0cec708264197bd993cbed085f442" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:50.036152 22600 consensus_queue.cc:1035] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181852bc61e24d4387a01f2b9527663a" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:37:50.043681 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.041369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:36763 (local address 127.10.207.2:38545)
0807 08:37:50.041416 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:50.041419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:50.041427 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:50.041495 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:50.041496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:50.041508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:50.041562 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:37:50.041663 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.042116 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.042926 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.043167 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.043237 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:50.043432 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:37:50.043466 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:37:50.043524 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:50.043609 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:37:50.043633 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:37:50.043740 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.041327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:38545 (local address 127.10.207.1:36763)
0807 08:37:50.041381 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:50.041392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:50.041433 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:50.041572 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:50.041572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:50.041574 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:37:50.041616 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:50.041618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.042209 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.042211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:50.042839 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:50.042842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:50.043263 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:50.043264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:50.043342 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:50.043353 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:37:50.043547 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:37:50.043549 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:37:50.043604 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:37:50.043671 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:37:50.053231 21804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:50.053335 21804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:37:50.054180 21804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2dbebe4088104f54a39d5772186b3660: Sending DeleteTablet for 3 replicas of tablet 6e3db7521ae04ee8a793ebcde6d27739
I20260807 08:37:50.054510 22025 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3db7521ae04ee8a793ebcde6d27739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:50 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53738
I20260807 08:37:50.054517 22469 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3db7521ae04ee8a793ebcde6d27739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:50 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53188
I20260807 08:37:50.054638 22623 tablet_replica.cc:330] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: stopping tablet replica
I20260807 08:37:50.054638 22624 tablet_replica.cc:330] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442: stopping tablet replica
I20260807 08:37:50.054684 22623 raft_consensus.cc:2229] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:50.054688 22624 raft_consensus.cc:2229] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:37:50.054720 22623 raft_consensus.cc:2258] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:50.054725 22624 raft_consensus.cc:2258] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:50.054900 22251 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3db7521ae04ee8a793ebcde6d27739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:37:50 UTC) from {username='oryx', principal='oryx/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46746
I20260807 08:37:50.055037 22621 tablet_replica.cc:330] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: stopping tablet replica
I20260807 08:37:50.055083 22623 ts_tablet_manager.cc:1905] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:50.055145 22621 raft_consensus.cc:2229] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:37:50.055222 22621 raft_consensus.cc:2258] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:37:50.055289 22624 ts_tablet_manager.cc:1905] T 6e3db7521ae04ee8a793ebcde6d27739 P 7aa0cec708264197bd993cbed085f442: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:50.055366 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 21923
I20260807 08:37:50.060030 22623 ts_tablet_manager.cc:1918] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:50.060078 22623 log.cc:1196] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/6e3db7521ae04ee8a793ebcde6d27739
I20260807 08:37:50.060257 22623 ts_tablet_manager.cc:1939] T 6e3db7521ae04ee8a793ebcde6d27739 P 181852bc61e24d4387a01f2b9527663a: Deleting consensus metadata
I20260807 08:37:50.060528 21784 catalog_manager.cc:4841] TS 181852bc61e24d4387a01f2b9527663a (127.10.207.3:39141): tablet 6e3db7521ae04ee8a793ebcde6d27739 (table test-table [id=514f8ecdcc3241ef9790c8f1b3ec264e]) successfully deleted
I20260807 08:37:50.066075 22621 ts_tablet_manager.cc:1905] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:37:50.067611 22621 ts_tablet_manager.cc:1918] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:37:50.067672 22621 log.cc:1196] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/6e3db7521ae04ee8a793ebcde6d27739
I20260807 08:37:50.067924 22621 ts_tablet_manager.cc:1939] T 6e3db7521ae04ee8a793ebcde6d27739 P 213d0af2d18649d0afaf6b08f283d9c9: Deleting consensus metadata
I20260807 08:37:50.068325 21782 catalog_manager.cc:4841] TS 213d0af2d18649d0afaf6b08f283d9c9 (127.10.207.2:38545): tablet 6e3db7521ae04ee8a793ebcde6d27739 (table test-table [id=514f8ecdcc3241ef9790c8f1b3ec264e]) successfully deleted
W20260807 08:37:50.072980 21784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:42709 (error 108)
I20260807 08:37:50.073328 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 22149
I20260807 08:37:50.081125 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:50.077278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:42709 (local address 127.0.0.1:41850)
0807 08:37:50.080999 (+ 3721us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:50.081068 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3693,"thread_start_us":79,"threads_started":1}
W20260807 08:37:50.081243 21784 catalog_manager.cc:4584] TS 7aa0cec708264197bd993cbed085f442 (127.10.207.1:42709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e3db7521ae04ee8a793ebcde6d27739: Network error: Client connection negotiation failed: client connection to 127.10.207.1:42709: connect: Connection refused (error 111)
I20260807 08:37:50.089249 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 22369
I20260807 08:37:50.102218 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 21754
2026-08-07T08:37:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3507 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:50 dist-test-slave-h6n0 krb5kdc[22715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:50 dist-test-slave-h6n0 krb5kdc[22715](info): set up 2 sockets
Aug 07 08:37:50 dist-test-slave-h6n0 krb5kdc[22715](info): commencing operation
krb5kdc: starting...
W20260807 08:37:51.388963 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:51 dist-test-slave-h6n0 krb5kdc[22715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:51.525731 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45133
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:45133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:51.617868 22736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:51.618104 22736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:51.618130 22736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:51.619237 22736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:51.619306 22736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:51.619323 22736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:51.619338 22736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:51.619350 22736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:51.620814 22736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:45133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:51.621027 22736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:51.621282 22736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:51.626631 22743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:51.626631 22742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:51.626657 22745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:51.626974 22736 server_base.cc:1034] running on GCE node
I20260807 08:37:51.627235 22736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:51.627502 22736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:51.628654 22736 hybrid_clock.cc:648] HybridClock initialized: now 1786091871628627 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T08:37:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1525 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:37:51 dist-test-slave-h6n0 krb5kdc[22751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:37:51 dist-test-slave-h6n0 krb5kdc[22751](info): set up 2 sockets
Aug 07 08:37:51 dist-test-slave-h6n0 krb5kdc[22751](info): commencing operation
krb5kdc: starting...
W20260807 08:37:52.894778 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:37:52 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:37:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:37:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:37:53.028597 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43685
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:53.119958 22768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:53.120204 22768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:53.120229 22768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:53.121456 22768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:53.121508 22768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:53.121523 22768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:53.121536 22768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:53.121549 22768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:53.123158 22768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:53.123448 22768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:53.123708 22768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:53.129300 22777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.129362 22778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.129292 22780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.129582 22768 server_base.cc:1034] running on GCE node
I20260807 08:37:53.129949 22768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:53.130483 22768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:53.131740 22768 hybrid_clock.cc:648] HybridClock initialized: now 1786091873131711 us; error 64 us; skew 500 ppm
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:53.134723 22768 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:53.135183 22768 webserver.cc:460] Webserver started at http://127.10.207.62:36003/ using document root <none> and password file <none>
I20260807 08:37:53.135437 22768 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:53.135488 22768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:53.135586 22768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:53.136721 22768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "b5632a0f69e24c31a74f336bb663702d"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "23afe219042630287413a4a7f5a51b82"
server_key_iv: "de64bdd142b1c9c178d635c4a4068566"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.137064 22768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "b5632a0f69e24c31a74f336bb663702d"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "23afe219042630287413a4a7f5a51b82"
server_key_iv: "de64bdd142b1c9c178d635c4a4068566"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.138523 22768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:37:53.139441 22792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.139706 22768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:53.139796 22768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "b5632a0f69e24c31a74f336bb663702d"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "23afe219042630287413a4a7f5a51b82"
server_key_iv: "de64bdd142b1c9c178d635c4a4068566"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.139904 22768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:53.152135 22768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:53.154445 22768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:53.154700 22768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:53.159575 22768 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43685
I20260807 08:37:53.159642 22869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43685 every 8 connection(s)
I20260807 08:37:53.160000 22768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:53.161046 22871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:37:53.163949 22871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Bootstrap starting.
I20260807 08:37:53.164548 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 22768
I20260807 08:37:53.164690 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:37:53.164716 22871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Neither blocks nor log segments found. Creating new log.
I20260807 08:37:53.164878 11068 external_mini_cluster.cc:1428] Setting key 0985c8332e0c1a025e398e8ddf8f31a8
I20260807 08:37:53.165091 22871 log.cc:824] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:53.165958 22871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: No bootstrap required, opened a new log
I20260807 08:37:53.168394 22871 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:53.168562 22871 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:37:53.168614 22871 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5632a0f69e24c31a74f336bb663702d, State: Initialized, Role: FOLLOWER
I20260807 08:37:53.168792 22871 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:53.168902 22871 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:53.168947 22871 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:53.168995 22871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:37:53.169682 22871 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:53.169808 22871 leader_election.cc:302] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5632a0f69e24c31a74f336bb663702d; no voters:
I20260807 08:37:53.170024 22871 leader_election.cc:288] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:37:53.170208 22877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:37:53.170341 22871 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:53.170387 22877 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 LEADER]: Becoming Leader. State: Replica: b5632a0f69e24c31a74f336bb663702d, State: Running, Role: LEADER
I20260807 08:37:53.170487 22877 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:53.170930 22879 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5632a0f69e24c31a74f336bb663702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } } }
I20260807 08:37:53.171023 22879 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:53.171245 22881 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5632a0f69e24c31a74f336bb663702d. Latest consensus state: current_term: 1 leader_uuid: "b5632a0f69e24c31a74f336bb663702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } } }
I20260807 08:37:53.171332 22881 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:53.171476 22890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:53.172041 22890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:53.177433 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.165404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.10.207.62:43685)
0807 08:37:53.165591 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:53.165603 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:53.165785 (+ 182us) server_negotiation.cc:407] Connection header received
0807 08:37:53.166428 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:53.166436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:53.166489 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:53.166690 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:53.167378 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.168369 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.169354 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.169836 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.169954 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.173411 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:53.173415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:53.173415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:53.173431 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:53.175675 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.176059 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.176062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.176062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.176188 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.176400 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.176401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.176401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.176734 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:53.176946 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:53.177018 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:37:53.177187 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260807 08:37:53.178407 22890 catalog_manager.cc:1347] Generated new cluster ID: d71283cfdade4b39958a449758075198
I20260807 08:37:53.178476 22890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:53.195209 22890 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:37:53.195866 22890 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:53.217540 22890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Generated new TSK 0
I20260807 08:37:53.217810 22890 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:37:53.257421 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43685
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:53.367093 22914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:53.367314 22914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:53.367338 22914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:53.368526 22914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:53.368575 22914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:53.368603 22914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:53.370101 22914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:53.370404 22914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:53.370720 22914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:53.376443 22920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.376451 22924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.376492 22921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.376900 22914 server_base.cc:1034] running on GCE node
I20260807 08:37:53.377151 22914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:53.377409 22914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:53.378592 22914 hybrid_clock.cc:648] HybridClock initialized: now 1786091873378568 us; error 52 us; skew 500 ppm
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:53.381362 22914 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:37:53.381834 22914 webserver.cc:460] Webserver started at http://127.10.207.1:34465/ using document root <none> and password file <none>
I20260807 08:37:53.382133 22914 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:53.382185 22914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:53.382357 22914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:53.383582 22914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "b3a3eedf2f954338a1924adf6b4f033e"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "7f17c70c823d024cd60fa7674249db13"
server_key_iv: "e9fbde17e48944da7311bfd5ab942e72"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.383971 22914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "b3a3eedf2f954338a1924adf6b4f033e"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "7f17c70c823d024cd60fa7674249db13"
server_key_iv: "e9fbde17e48944da7311bfd5ab942e72"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.385452 22914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:37:53.386364 22932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.386660 22914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:53.386741 22914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "b3a3eedf2f954338a1924adf6b4f033e"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "7f17c70c823d024cd60fa7674249db13"
server_key_iv: "e9fbde17e48944da7311bfd5ab942e72"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.386843 22914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:53.404577 22914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:53.406955 22914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:53.407142 22914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:53.407397 22914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:53.407737 22914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:53.407792 22914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.407824 22914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:53.407857 22914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.415766 22914 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:37859
I20260807 08:37:53.415856 23094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:37859 every 8 connection(s)
I20260807 08:37:53.416163 22914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:53.426074 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 22914
I20260807 08:37:53.426216 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:37:53.426460 11068 external_mini_cluster.cc:1428] Setting key 553ded26a8172866fc258d4d6863f139
I20260807 08:37:53.427912 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.417762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:59623 (local address 127.10.207.62:43685)
0807 08:37:53.417855 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:53.417858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:53.417873 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:37:53.418194 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:53.418196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:53.418207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:53.418278 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:53.419415 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.419937 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.421038 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.421369 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.421463 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.424044 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:53.424046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:53.424046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:53.424054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:53.425760 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.426410 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.426411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.426411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.426688 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.427102 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.427103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.427104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.427261 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:53.427352 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:53.427742 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 08:37:53.427839 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 08:37:53.428279 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.417047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:59623)
0807 08:37:53.417275 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:53.417336 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:53.418074 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:53.418344 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:53.418347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:53.419067 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:53.419318 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.419323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.420038 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.420038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.420928 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.420930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.421477 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.421478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.421588 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.422208 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:53.422217 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:53.423903 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:53.425864 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.425874 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.425888 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.426267 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.426854 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.426855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.426855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.427013 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.427348 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:53.427355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:53.427546 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 08:37:53.427691 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:37:53.427895 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":93,"threads_started":1}
I20260807 08:37:53.430248 23095 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43685
I20260807 08:37:53.430377 23095 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:53.430661 23095 heartbeater.cc:511] Master 127.10.207.62:43685 requested a full tablet report, sending...
I20260807 08:37:53.431348 22818 ts_manager.cc:194] Registered new tserver with Master: b3a3eedf2f954338a1924adf6b4f033e (127.10.207.1:37859)
I20260807 08:37:53.432241 22818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:59623
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:37:53.457331 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43685
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:53.549275 23105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:53.549571 23105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:53.549621 23105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:53.551456 23105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:53.551543 23105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:53.551596 23105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:53.553472 23105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:53.553794 23105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:53.554072 23105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:53.559772 23116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.559990 23112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.560047 23105 server_base.cc:1034] running on GCE node
W20260807 08:37:53.560187 23111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.560499 23105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:53.560860 23105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:53.562090 23105 hybrid_clock.cc:648] HybridClock initialized: now 1786091873562070 us; error 52 us; skew 500 ppm
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:53.565418 23105 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:37:53.565970 23105 webserver.cc:460] Webserver started at http://127.10.207.2:39273/ using document root <none> and password file <none>
I20260807 08:37:53.566251 23105 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:53.566334 23105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:53.566499 23105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:53.567667 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "f2a86bb9669a43018326c27fd49e9303"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "8197b5924ea2ad45229a56faa5574cf9"
server_key_iv: "08bbdadf953cedb64c18b5ec01444c9a"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.568097 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "f2a86bb9669a43018326c27fd49e9303"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "8197b5924ea2ad45229a56faa5574cf9"
server_key_iv: "08bbdadf953cedb64c18b5ec01444c9a"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.569582 23105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:37:53.570483 23128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.570746 23105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:53.570861 23105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "f2a86bb9669a43018326c27fd49e9303"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "8197b5924ea2ad45229a56faa5574cf9"
server_key_iv: "08bbdadf953cedb64c18b5ec01444c9a"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.570966 23105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:53.599439 23105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:53.601759 23105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:53.602046 23105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:53.602332 23105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:53.602692 23105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:53.602753 23105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.602787 23105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:53.602819 23105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.610076 23105 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:35349
I20260807 08:37:53.610139 23297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:35349 every 8 connection(s)
I20260807 08:37:53.610457 23105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:53.614708 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 23105
I20260807 08:37:53.614820 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:37:53.615064 11068 external_mini_cluster.cc:1428] Setting key abbd9fb86488876f08b07cd08f7d66d3
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:53.622241 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.611373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:38647)
0807 08:37:53.611597 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:53.611656 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:53.612243 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:53.612585 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:53.612589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:53.613280 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:53.613514 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.613519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.614234 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.614234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.615404 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.615407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.615848 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.615850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.615998 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.616655 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:53.616665 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:53.618514 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:53.620651 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.620658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.620660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.620915 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.621176 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.621177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.621177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.621279 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.621714 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:53.621718 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:53.621840 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 08:37:53.621937 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:37:53.622080 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":135,"threads_started":1}
I20260807 08:37:53.622274 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.611443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:38647 (local address 127.10.207.62:43685)
0807 08:37:53.611491 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:53.611503 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:53.611826 (+ 323us) server_negotiation.cc:407] Connection header received
0807 08:37:53.612385 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:53.612386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:53.612399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:53.612494 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:53.613623 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.614129 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.615509 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.615739 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.615835 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.618678 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:53.618680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:53.618681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:53.618689 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:53.620548 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.621004 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.621006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.621006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.621084 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.621452 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.621453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.621454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.621632 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:53.621723 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:53.621993 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 08:37:53.622095 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:37:53.622910 23298 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43685
I20260807 08:37:53.623042 23298 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:53.623239 23298 heartbeater.cc:511] Master 127.10.207.62:43685 requested a full tablet report, sending...
I20260807 08:37:53.623750 22818 ts_manager.cc:194] Registered new tserver with Master: f2a86bb9669a43018326c27fd49e9303 (127.10.207.2:35349)
I20260807 08:37:53.624328 22818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:38647
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:37:53.645088 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43685
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:53.738037 23305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:53.738273 23305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:53.738299 23305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:53.739476 23305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:53.739521 23305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:53.739547 23305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:53.741077 23305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:53.741294 23305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:53.741498 23305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:53.747249 23312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.747243 23311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:53.747243 23314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:53.748056 23305 server_base.cc:1034] running on GCE node
I20260807 08:37:53.748282 23305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:53.748517 23305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:53.749675 23305 hybrid_clock.cc:648] HybridClock initialized: now 1786091873749659 us; error 36 us; skew 500 ppm
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:53.752538 23305 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:37:53.753027 23305 webserver.cc:460] Webserver started at http://127.10.207.3:42063/ using document root <none> and password file <none>
I20260807 08:37:53.753309 23305 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:53.753379 23305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:53.753539 23305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:37:53.754727 23305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "29150903d5fe43acb15c31d8681f6aab"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "1b2a533d87d7e6162878b60285c9924e"
server_key_iv: "af1d3743263f48b3c28009494f881275"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.755115 23305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "29150903d5fe43acb15c31d8681f6aab"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "1b2a533d87d7e6162878b60285c9924e"
server_key_iv: "af1d3743263f48b3c28009494f881275"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.756645 23305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:37:53.757781 23325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.758031 23305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:53.758144 23305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "29150903d5fe43acb15c31d8681f6aab"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "1b2a533d87d7e6162878b60285c9924e"
server_key_iv: "af1d3743263f48b3c28009494f881275"
server_key_version: "encryptionkey@0"
I20260807 08:37:53.758239 23305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:53.784950 23305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:53.787405 23305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:53.787587 23305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:53.787858 23305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:53.788221 23305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:53.788280 23305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.788316 23305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:53.788345 23305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:53.795758 23305 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:45233
I20260807 08:37:53.795910 23489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:45233 every 8 connection(s)
I20260807 08:37:53.796204 23305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:37:53.801865 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 23305
I20260807 08:37:53.801996 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:37:53.802224 11068 external_mini_cluster.cc:1428] Setting key 31007917adfdcc3c02529c28afe3b864
Aug 07 08:37:53 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091873, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:37:53.807335 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.797145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:41873 (local address 127.10.207.62:43685)
0807 08:37:53.797247 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:53.797249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:53.797380 (+ 131us) server_negotiation.cc:407] Connection header received
0807 08:37:53.797897 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:53.797899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:53.797911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:53.798020 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:53.799344 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.799906 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.801167 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.801368 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.801435 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.804297 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:53.804298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:53.804298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:53.804305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:53.805803 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.806218 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.806219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.806220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.806287 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:53.806628 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:53.806629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:53.806630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:53.806777 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:53.806849 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:53.807180 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 08:37:53.807258 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:37:53.807475 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:53.797042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:41873)
0807 08:37:53.797226 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:53.797268 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:53.797778 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:53.798191 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:53.798197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:53.798973 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:53.799219 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.799224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.800037 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.800038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.801055 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:53.801058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:53.801496 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:53.801497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:53.801669 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:53.802408 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:53.802436 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:53.804179 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:37:53.805896 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.805902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.805904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.806121 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.806363 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:37:53.806364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:37:53.806364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:37:53.806467 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:37:53.806885 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:37:53.806892 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:37:53.807018 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 08:37:53.807128 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 08:37:53.807265 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260807 08:37:53.808174 23490 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43685
I20260807 08:37:53.808336 23490 heartbeater.cc:464] Registering TS with master...
I20260807 08:37:53.808566 23490 heartbeater.cc:511] Master 127.10.207.62:43685 requested a full tablet report, sending...
I20260807 08:37:53.809041 22818 ts_manager.cc:194] Registered new tserver with Master: 29150903d5fe43acb15c31d8681f6aab (127.10.207.3:45233)
I20260807 08:37:53.809484 22818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:41873
I20260807 08:37:53.814502 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab.
Entry for principal oryx/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab.
WARNING: no policy specified for oryx/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab.
Entry for principal oryx/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab.
WARNING: no policy specified for oryx/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab.
Entry for principal oryx/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab.
I20260807 08:37:53.892287 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 22914
I20260807 08:37:53.910238 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 23105
I20260807 08:37:53.919445 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 23305
I20260807 08:37:53.930642 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 22768
I20260807 08:37:53.940387 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43685
--webserver_interface=127.10.207.62
--webserver_port=36003
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:37:54.033156 23499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:54.033394 23499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:54.033418 23499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:54.034621 23499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:37:54.034669 23499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:54.034683 23499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:54.034698 23499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:37:54.034709 23499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:37:54.036273 23499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=36003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:54.036552 23499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:54.036825 23499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:54.042145 23508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.042162 23511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:54.042306 23499 server_base.cc:1034] running on GCE node
W20260807 08:37:54.042146 23507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:54.042721 23499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:54.043052 23499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:54.044270 23499 hybrid_clock.cc:648] HybridClock initialized: now 1786091874044249 us; error 33 us; skew 500 ppm
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091874, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:54.047242 23499 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:37:54.047797 23499 webserver.cc:460] Webserver started at http://127.10.207.62:36003/ using document root <none> and password file <none>
I20260807 08:37:54.048059 23499 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:54.048162 23499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:54.049985 23499 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 08:37:54.051282 23522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.051550 23499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:54.051664 23499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "b5632a0f69e24c31a74f336bb663702d"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "23afe219042630287413a4a7f5a51b82"
server_key_iv: "de64bdd142b1c9c178d635c4a4068566"
server_key_version: "encryptionkey@0"
I20260807 08:37:54.052153 23499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:54.072106 23499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:54.074141 23499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:54.074260 23499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:54.079748 23499 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43685
I20260807 08:37:54.079833 23596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43685 every 8 connection(s)
I20260807 08:37:54.080267 23499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:37:54.081961 23597 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:37:54.082701 23597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Bootstrap starting.
I20260807 08:37:54.086330 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 23499
I20260807 08:37:54.086745 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:37859
--local_ip_for_outbound_sockets=127.10.207.1
--tserver_master_addrs=127.10.207.62:43685
--webserver_port=34465
--webserver_interface=127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:37:54.092726 23597 log.cc:824] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:37:54.095304 23597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:37:54.095664 23597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Bootstrap complete.
I20260807 08:37:54.100227 23597 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:54.100625 23597 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5632a0f69e24c31a74f336bb663702d, State: Initialized, Role: FOLLOWER
I20260807 08:37:54.100838 23597 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:54.100929 23597 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:37:54.100991 23597 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:37:54.101042 23597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:37:54.102329 23597 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:54.102442 23597 leader_election.cc:302] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5632a0f69e24c31a74f336bb663702d; no voters:
I20260807 08:37:54.102666 23597 leader_election.cc:288] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:37:54.102834 23604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:37:54.102949 23597 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:37:54.103060 23604 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [term 2 LEADER]: Becoming Leader. State: Replica: b5632a0f69e24c31a74f336bb663702d, State: Running, Role: LEADER
I20260807 08:37:54.103190 23604 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } }
I20260807 08:37:54.103525 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b5632a0f69e24c31a74f336bb663702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } } }
I20260807 08:37:54.103662 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:54.104067 23607 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5632a0f69e24c31a74f336bb663702d. Latest consensus state: current_term: 2 leader_uuid: "b5632a0f69e24c31a74f336bb663702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5632a0f69e24c31a74f336bb663702d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43685 } } }
I20260807 08:37:54.104133 23616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:37:54.104154 23607 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d [sys.catalog]: This master's current role is: LEADER
I20260807 08:37:54.104981 23616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:37:54.105134 23616 catalog_manager.cc:1259] Loaded cluster ID: d71283cfdade4b39958a449758075198
I20260807 08:37:54.105170 23616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:37:54.106386 23616 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:37:54.106619 23616 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b5632a0f69e24c31a74f336bb663702d: Loaded TSK: 0
I20260807 08:37:54.106858 23616 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:37:54.180778 23599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:54.181015 23599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:54.181056 23599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:54.182189 23599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:54.182257 23599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:54.182304 23599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:37:54.183871 23599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.1.keytab
--principal=oryx/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:37859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=34465
--tserver_master_addrs=127.10.207.62:43685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:54.184132 23599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:54.184363 23599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:54.189545 23643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.189561 23636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.189549 23638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:54.189793 23599 server_base.cc:1034] running on GCE node
I20260807 08:37:54.190084 23599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:54.190382 23599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:54.191560 23599 hybrid_clock.cc:648] HybridClock initialized: now 1786091874191542 us; error 62 us; skew 500 ppm
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091874, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:54.194407 23599 init.cc:377] Logged in from keytab as oryx/127.10.207.1@KRBTEST.COM (short username oryx)
I20260807 08:37:54.194926 23599 webserver.cc:460] Webserver started at http://127.10.207.1:34465/ using document root <none> and password file <none>
I20260807 08:37:54.195161 23599 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:54.195245 23599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:54.196590 23599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:54.197221 23653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.197420 23599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:37:54.197537 23599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "b3a3eedf2f954338a1924adf6b4f033e"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "7f17c70c823d024cd60fa7674249db13"
server_key_iv: "e9fbde17e48944da7311bfd5ab942e72"
server_key_version: "encryptionkey@0"
I20260807 08:37:54.197863 23599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:54.206904 23599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:54.209015 23599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:54.209159 23599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:54.209401 23599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:54.209764 23599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:54.209805 23599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.209824 23599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:54.209837 23599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.217638 23599 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:37859
I20260807 08:37:54.217824 23802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:37859 every 8 connection(s)
I20260807 08:37:54.218096 23599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:37:54.224319 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 23599
I20260807 08:37:54.224658 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:35349
--local_ip_for_outbound_sockets=127.10.207.2
--tserver_master_addrs=127.10.207.62:43685
--webserver_port=39273
--webserver_interface=127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.229557 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.218825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:52319)
0807 08:37:54.219041 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.219100 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.219847 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.220346 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.220351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.221215 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.221528 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.221535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.222808 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.222809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.224144 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.224147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.225125 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.225126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.225280 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.225945 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.225956 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.229318 (+ 3362us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
W20260807 08:37:54.229725 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.219018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:52319 (local address 127.10.207.62:43685)
0807 08:37:54.219231 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.219240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.219280 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:37:54.220070 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.220076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.220119 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.220233 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.221687 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.222700 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.224345 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.225018 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.225131 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.229544 (+ 4413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:52319: BlockingRecv error: recv got EOF from 127.10.207.1:52319 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":120,"threads_started":1}
W20260807 08:37:54.229851 23803 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:43685 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.207.62:43685: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.236333 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.230071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:34411)
0807 08:37:54.230118 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.230129 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.230170 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.230394 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.230395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.230611 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.230682 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.230685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.231662 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.231663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.232607 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.232610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.233166 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.233168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.233252 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.233818 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.233824 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.236247 (+ 2423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 08:37:54.236439 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.230169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:34411 (local address 127.10.207.62:43685)
0807 08:37:54.230222 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.230224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.230234 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:54.230263 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.230264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.230280 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.230363 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.230782 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.231556 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.232724 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.233063 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.233154 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.236366 (+ 3212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:34411: BlockingRecv error: recv got EOF from 127.10.207.1:34411 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.242291 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.236941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:42349)
0807 08:37:54.236998 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.237008 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.237043 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.237229 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.237230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.237403 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.237444 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.237446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.238438 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.238440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.239267 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.239269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.239777 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.239778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.239834 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.240323 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.240328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.242226 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 08:37:54.242383 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.237028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:42349 (local address 127.10.207.62:43685)
0807 08:37:54.237079 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.237081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.237091 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:54.237119 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.237120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.237136 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.237216 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.237537 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.238333 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.239362 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.239696 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.239768 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.242331 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:42349: BlockingRecv error: recv got EOF from 127.10.207.1:42349 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 08:37:54.242758 23803 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:37:54.320801 23812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:54.321013 23812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:54.321053 23812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:54.322165 23812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:54.322232 23812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:54.322276 23812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:37:54.323750 23812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.2.keytab
--principal=oryx/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:35349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=39273
--tserver_master_addrs=127.10.207.62:43685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:54.324018 23812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:54.324240 23812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:54.329361 23818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.329375 23819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.329435 23821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:54.329677 23812 server_base.cc:1034] running on GCE node
I20260807 08:37:54.329967 23812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:54.330226 23812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:54.331416 23812 hybrid_clock.cc:648] HybridClock initialized: now 1786091874331388 us; error 57 us; skew 500 ppm
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091874, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:54.334440 23812 init.cc:377] Logged in from keytab as oryx/127.10.207.2@KRBTEST.COM (short username oryx)
I20260807 08:37:54.334990 23812 webserver.cc:460] Webserver started at http://127.10.207.2:39273/ using document root <none> and password file <none>
I20260807 08:37:54.335247 23812 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:54.335337 23812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:54.336772 23812 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:54.337457 23830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.337682 23812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:54.337812 23812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "f2a86bb9669a43018326c27fd49e9303"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "8197b5924ea2ad45229a56faa5574cf9"
server_key_iv: "08bbdadf953cedb64c18b5ec01444c9a"
server_key_version: "encryptionkey@0"
I20260807 08:37:54.338136 23812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:54.346514 23812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:54.348387 23812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:54.348498 23812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:54.348663 23812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:54.348974 23812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:54.349035 23812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.349068 23812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:54.349119 23812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.356142 23812 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:35349
I20260807 08:37:54.356268 23978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:35349 every 8 connection(s)
I20260807 08:37:54.356544 23812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:37:54.361480 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 23812
I20260807 08:37:54.361788 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:45233
--local_ip_for_outbound_sockets=127.10.207.3
--tserver_master_addrs=127.10.207.62:43685
--webserver_port=42063
--webserver_interface=127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.365455 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.357346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:57753)
0807 08:37:54.357552 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.357623 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.358257 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.358699 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.358703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.359348 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.359587 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.359600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.360312 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.360313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.361290 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.361294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.361824 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.361825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.362025 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.362695 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.362705 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.365202 (+ 2497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
W20260807 08:37:54.365473 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.357396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:57753 (local address 127.10.207.62:43685)
0807 08:37:54.357505 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.357509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.357734 (+ 225us) server_negotiation.cc:407] Connection header received
0807 08:37:54.358448 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.358450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.358476 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.358600 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.359692 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.360179 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.361397 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.361718 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.361792 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.365404 (+ 3612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:57753: BlockingRecv error: recv got EOF from 127.10.207.2:57753 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 08:37:54.366151 23980 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:43685 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.207.62:43685: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.370972 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.366532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:40639)
0807 08:37:54.366626 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.366657 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.366703 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.366855 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.366856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.366996 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.367048 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.367049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.367722 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.367723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.368244 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.368246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.368611 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.368612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.368685 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.369172 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.369177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.370885 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 08:37:54.371028 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.366607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:40639 (local address 127.10.207.62:43685)
0807 08:37:54.366684 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.366686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.366696 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:54.366768 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.366769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.366780 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.366850 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.367133 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.367635 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.368330 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.368529 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.368610 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.370959 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:40639: BlockingRecv error: recv got EOF from 127.10.207.2:40639 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.376252 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.371320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:41435)
0807 08:37:54.371363 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.371374 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.371411 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.371632 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.371632 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.371743 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.371770 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.371771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.372783 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.372784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.373293 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.373295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.373766 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.373766 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.373804 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.374304 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.374309 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.376204 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 08:37:54.376381 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.371376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:41435 (local address 127.10.207.62:43685)
0807 08:37:54.371459 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.371462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.371474 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:37:54.371499 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.371500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.371514 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.371599 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.371857 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.372680 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.373383 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.373684 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.373759 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.376312 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:41435: BlockingRecv error: recv got EOF from 127.10.207.2:41435 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 08:37:54.376410 23980 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:37:54.446292 23983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:37:54.446480 23983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:37:54.446503 23983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:37:54.447722 23983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:37:54.447770 23983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:37:54.447798 23983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:37:54.449275 23983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/krb5kdc/oryx_127.10.207.3.keytab
--principal=oryx/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:45233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=42063
--tserver_master_addrs=127.10.207.62:43685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:37:54.449501 23983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:37:54.449733 23983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:37:54.455109 23989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.455137 23992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:37:54.455194 23990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:37:54.455307 23983 server_base.cc:1034] running on GCE node
I20260807 08:37:54.455677 23983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:37:54.455914 23983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:37:54.457067 23983 hybrid_clock.cc:648] HybridClock initialized: now 1786091874457045 us; error 42 us; skew 500 ppm
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091874, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:37:54.463145 23983 init.cc:377] Logged in from keytab as oryx/127.10.207.3@KRBTEST.COM (short username oryx)
I20260807 08:37:54.463639 23983 webserver.cc:460] Webserver started at http://127.10.207.3:42063/ using document root <none> and password file <none>
I20260807 08:37:54.463868 23983 fs_manager.cc:359] Metadata directory not provided
I20260807 08:37:54.463959 23983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:37:54.465329 23983 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:54.466002 24003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.466214 23983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:37:54.466324 23983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "29150903d5fe43acb15c31d8681f6aab"
format_stamp: "Formatted at 2026-08-07 08:37:53 on dist-test-slave-h6n0"
server_key: "1b2a533d87d7e6162878b60285c9924e"
server_key_iv: "af1d3743263f48b3c28009494f881275"
server_key_version: "encryptionkey@0"
I20260807 08:37:54.466693 23983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:37:54.480472 23983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:37:54.482761 23983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:37:54.482928 23983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:37:54.483155 23983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:37:54.483470 23983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:37:54.483530 23983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.483573 23983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:37:54.483601 23983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:37:54.490777 23983 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:45233
I20260807 08:37:54.490866 24161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:45233 every 8 connection(s)
I20260807 08:37:54.491216 23983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.497781 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 23983
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:37:54.499122 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.492106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:40285 (local address 127.10.207.62:43685)
0807 08:37:54.492174 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.492176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.492409 (+ 233us) server_negotiation.cc:407] Connection header received
0807 08:37:54.492933 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.492935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.492948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.493000 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.493903 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.494395 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.495466 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.495664 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.495751 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.499066 (+ 3315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:40285: BlockingRecv error: recv got EOF from 127.10.207.3:40285 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:37:54.499189 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.492091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:40285)
0807 08:37:54.492284 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.492320 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.492815 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.493069 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.493072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.493614 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.493793 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.493797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.494508 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.494509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.495359 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.495362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.495763 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.495764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.495879 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.496391 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.496399 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.499000 (+ 2601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
W20260807 08:37:54.499423 24162 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:43685 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.207.62:43685: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.505510 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.499644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:44815)
0807 08:37:54.499862 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.499872 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.499912 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.500052 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.500053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.500182 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.500220 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.500221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.501099 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.501100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.501826 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.501829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.502340 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.502341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.502437 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.502923 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.502928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.505429 (+ 2501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":106,"threads_started":1}
W20260807 08:37:54.505587 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.499859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:44815 (local address 127.10.207.62:43685)
0807 08:37:54.499918 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.499921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.499947 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:37:54.499966 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.499967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.499976 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.500049 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.500308 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.501005 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.501932 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.502229 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.502292 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.505524 (+ 3232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:44815: BlockingRecv error: recv got EOF from 127.10.207.3:44815 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:37:54.507762 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.498453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55768 (local address 127.10.207.62:43685)
0807 08:37:54.498624 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.498627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.498743 (+ 116us) server_negotiation.cc:407] Connection header received
0807 08:37:54.498819 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.498820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.498833 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.498923 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.499765 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.500405 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.501432 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.501778 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.501861 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.503494 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:37:54.503501 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:37:54.503502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:37:54.503518 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:37:54.505971 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:54.506480 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:54.506484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:54.506484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:54.506653 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:37:54.506901 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:37:54.506902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:37:54.506902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:37:54.507226 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:37:54.507376 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:37:54.507538 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 08:37:54.507650 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":120,"threads_started":1}
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:54 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:54.511250 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:54.505962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:47723)
0807 08:37:54.506013 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:54.506024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:54.506060 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:54.506413 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:54.506415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:54.506624 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:54.506662 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.506665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.507546 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.507547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.508336 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:54.508339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.508808 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.508809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:54.508867 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.509322 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:54.509327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:54.511164 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:37:54.511298 23811 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:54.506213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:47723 (local address 127.10.207.62:43685)
0807 08:37:54.506273 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:54.506274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:54.506282 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:37:54.506301 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:54.506302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:54.506311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:54.506365 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:54.506768 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.507440 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.508443 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:54.508706 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:54.508792 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:54.511236 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:47723: BlockingRecv error: recv got EOF from 127.10.207.3:47723 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 08:37:54.511472 24162 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 08:37:55 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:55 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:55.248818 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:55.243189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:34091)
0807 08:37:55.243367 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:55.243379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:55.243437 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:55.243637 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:55.243638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:55.243909 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:55.243972 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:55.243975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.244759 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.244761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:55.245343 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:55.245344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.245774 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.245775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:55.245821 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:55.246364 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:55.246368 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:55.248719 (+ 2351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
W20260807 08:37:55.248955 24174 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:55.243287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:34091 (local address 127.10.207.62:43685)
0807 08:37:55.243454 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:55.243456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:55.243496 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:37:55.243525 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:55.243525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:55.243537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:55.243608 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:55.244075 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.244652 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.245459 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.245685 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.245765 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:55.248831 (+ 3066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:34091: BlockingRecv error: recv got EOF from 127.10.207.1:34091 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
Aug 07 08:37:55 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:55 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:55.381635 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:55.376732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:54861)
0807 08:37:55.376897 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:55.376909 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:55.376972 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:55.377148 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:55.377149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:55.377321 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:55.377375 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:55.377377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.378143 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.378143 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:55.378790 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:55.378792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.379153 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.379153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:55.379193 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:55.379689 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:55.379694 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:55.381574 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
W20260807 08:37:55.381803 24174 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:55.376846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:54861 (local address 127.10.207.62:43685)
0807 08:37:55.376924 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:55.376926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:55.376964 (+ 38us) server_negotiation.cc:407] Connection header received
0807 08:37:55.377046 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:55.377047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:55.377062 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:55.377130 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:55.377488 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.378050 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.378875 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.379076 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.379139 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:55.381704 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:54861: BlockingRecv error: recv got EOF from 127.10.207.2:54861 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 08:37:55 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:55 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:55.517795 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:55.511961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:39293)
0807 08:37:55.512180 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:55.512194 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:55.512283 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:55.512478 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:55.512479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:55.512691 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:55.512764 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:55.512765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.513527 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.513528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:55.514173 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:55.514175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.514600 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.514601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:55.514664 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:55.515407 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:55.515424 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:55.517684 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":154,"threads_started":1}
W20260807 08:37:55.517834 24174 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:55.512070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:39293 (local address 127.10.207.62:43685)
0807 08:37:55.512147 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:55.512154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:55.512288 (+ 134us) server_negotiation.cc:407] Connection header received
0807 08:37:55.512377 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:55.512377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:55.512391 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:55.512446 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:55.512899 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.513411 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.514266 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:55.514445 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:55.514505 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:55.517766 (+ 3261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:39293: BlockingRecv error: recv got EOF from 127.10.207.3:39293 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 08:37:56 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:56 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:56.255265 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:56.249438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:54149)
0807 08:37:56.249582 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:56.249596 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:56.249647 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:56.250051 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:56.250052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:56.250229 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:56.250287 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:56.250288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.251191 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.251193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:56.251765 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:56.251768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.252174 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.252175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:56.252228 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:56.252953 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:56.252959 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:56.255186 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
W20260807 08:37:56.255457 24184 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:56.249543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:54149 (local address 127.10.207.62:43685)
0807 08:37:56.249866 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:56.249868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:56.249880 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:37:56.249913 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:56.249913 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:56.249926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:56.250020 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:56.250380 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.251083 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.251866 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.252066 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.252165 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:56.255344 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:54149: BlockingRecv error: recv got EOF from 127.10.207.1:54149 (error 108)
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":243,"threads_started":1}
Aug 07 08:37:56 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:56 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:56.387482 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:56.382259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:52957)
0807 08:37:56.382448 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:56.382461 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:56.382560 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:56.382730 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:56.382749 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:56.383056 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:56.383186 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:56.383190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.383943 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.383943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:56.384533 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:56.384534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.384887 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.384888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:56.384930 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:56.385436 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:56.385441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:56.387416 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":121,"threads_started":1}
W20260807 08:37:56.387616 24184 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:56.382386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:52957 (local address 127.10.207.62:43685)
0807 08:37:56.382508 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:56.382515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:56.382567 (+ 52us) server_negotiation.cc:407] Connection header received
0807 08:37:56.382642 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:56.382642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:56.382653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:56.382704 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:56.383298 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.383857 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.384631 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.384814 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.384871 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:56.387526 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:52957: BlockingRecv error: recv got EOF from 127.10.207.2:52957 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 08:37:56 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:56 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:56.523794 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:56.518410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:43989)
0807 08:37:56.518654 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:56.518667 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:56.518727 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:56.518917 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:56.518918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:56.519096 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:56.519152 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:56.519153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.519932 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.519933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:56.520476 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:56.520477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.520883 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.520884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:56.520925 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:56.521446 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:56.521450 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:56.523735 (+ 2285us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":95,"threads_started":1}
W20260807 08:37:56.524058 24184 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:56.518621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:43989 (local address 127.10.207.62:43685)
0807 08:37:56.518689 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:56.518691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:56.518705 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:37:56.518809 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:56.518810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:56.518827 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:56.518889 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:56.519263 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.519825 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.520578 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:56.520799 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:56.520869 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:56.523980 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:43989: BlockingRecv error: recv got EOF from 127.10.207.3:43989 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 07 08:37:57 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:57 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:57.261291 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:57.255722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:42707)
0807 08:37:57.255860 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:57.255879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:57.255930 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:57.256269 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:57.256270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:57.256448 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:57.256513 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:57.256515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.257283 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.257284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:57.257838 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:57.257839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.258197 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.258198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:57.258241 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:57.258820 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:57.258825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:57.261167 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
W20260807 08:37:57.261344 24189 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:57.255861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:42707 (local address 127.10.207.62:43685)
0807 08:37:57.256051 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:57.256053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:57.256063 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:57.256104 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:57.256105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:57.256120 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:57.256214 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:57.256619 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.257199 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.257924 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.258125 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.258182 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:57.261260 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:42707: BlockingRecv error: recv got EOF from 127.10.207.1:42707 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
Aug 07 08:37:57 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:57 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:57.393635 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:57.388152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:43605)
0807 08:37:57.388399 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:57.388417 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:57.388550 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:57.388780 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:57.388781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:57.389012 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:57.389076 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:57.389078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.389949 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.389950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:57.390515 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:57.390517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.390985 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.390987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:57.391050 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:57.391590 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:57.391595 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:57.393501 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":141,"threads_started":1}
W20260807 08:37:57.393649 24189 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:57.388297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:43605 (local address 127.10.207.62:43685)
0807 08:37:57.388403 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:57.388406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:57.388521 (+ 115us) server_negotiation.cc:407] Connection header received
0807 08:37:57.388673 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:57.388674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:57.388690 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:57.388773 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:57.389162 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.389850 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.390686 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.390880 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.390986 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:57.393590 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:43605: BlockingRecv error: recv got EOF from 127.10.207.2:43605 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 08:37:57 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:57 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:57.529505 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:57.524282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:58967)
0807 08:37:57.524559 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:57.524595 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:57.524698 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:57.524876 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:57.524877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:57.525120 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:57.525181 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:57.525183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.525917 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.525918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:57.526464 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:57.526466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.526890 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.526891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:57.526949 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:57.527495 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:57.527500 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:57.529404 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":142,"threads_started":1}
W20260807 08:37:57.529656 24189 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:57.524419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:58967 (local address 127.10.207.62:43685)
0807 08:37:57.524545 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:57.524548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:57.524670 (+ 122us) server_negotiation.cc:407] Connection header received
0807 08:37:57.524795 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:57.524796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:57.524814 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:57.524865 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:57.525284 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.525832 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.526574 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:57.526798 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:57.526889 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:57.529539 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:58967: BlockingRecv error: recv got EOF from 127.10.207.3:58967 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 08:37:58 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:58 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:58.268149 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:58.261982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:52613)
0807 08:37:58.262182 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:58.262196 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:58.262273 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:58.262493 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:58.262495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:58.262766 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:58.262852 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:58.262854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.263697 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.263699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:58.264491 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:58.264494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.264942 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.264944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:58.265028 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:58.265687 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:58.265693 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:58.268023 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":123,"threads_started":1}
W20260807 08:37:58.268277 24193 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:58.262096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:52613 (local address 127.10.207.62:43685)
0807 08:37:58.262257 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:58.262259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:58.262271 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:37:58.262331 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:58.262332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:58.262345 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:58.262427 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:58.262968 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.263541 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.264590 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.264842 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.264936 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:58.268189 (+ 3253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:52613: BlockingRecv error: recv got EOF from 127.10.207.1:52613 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
Aug 07 08:37:58 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:58 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:58.400372 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:58.394310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:34267)
0807 08:37:58.394487 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:58.394505 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:58.394614 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:58.394831 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:58.394833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:58.395035 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:58.395142 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:58.395145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.395940 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.395942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:58.396598 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:58.396600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.397081 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.397083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:58.397214 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:58.397948 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:58.397955 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:58.400257 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
W20260807 08:37:58.400434 24193 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:58.394409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:34267 (local address 127.10.207.62:43685)
0807 08:37:58.394509 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:58.394511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:58.394598 (+ 87us) server_negotiation.cc:407] Connection header received
0807 08:37:58.394706 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:58.394707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:58.394720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:58.394787 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:58.395271 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.395839 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.396718 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.396976 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.397050 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:58.400362 (+ 3312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:34267: BlockingRecv error: recv got EOF from 127.10.207.2:34267 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 08:37:58 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:58 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:58.535487 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:58.530056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:42339)
0807 08:37:58.530268 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:58.530283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:58.530393 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:58.530692 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:58.530693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:58.530945 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:58.531001 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:58.531003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.531830 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.531831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:58.532376 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:58.532377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.532825 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.532826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:58.532869 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:58.533414 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:58.533419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:58.535429 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":116,"threads_started":1}
W20260807 08:37:58.535714 24193 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:58.530166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:42339 (local address 127.10.207.62:43685)
0807 08:37:58.530258 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:58.530261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:58.530399 (+ 138us) server_negotiation.cc:407] Connection header received
0807 08:37:58.530577 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:58.530578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:58.530597 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:58.530663 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:58.531117 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.531688 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.532500 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:58.532706 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:58.532779 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:58.535559 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:42339: BlockingRecv error: recv got EOF from 127.10.207.3:42339 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 08:37:59 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:59 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:59.275344 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:59.268751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:45763)
0807 08:37:59.268931 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:59.268947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:59.269018 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:59.269266 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:59.269267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:59.269489 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:59.269548 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:59.269550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.270556 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.270558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:59.271221 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:59.271223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.271760 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.271761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:59.271833 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:59.272831 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:59.272836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:59.275229 (+ 2393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
W20260807 08:37:59.275435 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:59.268820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:45763 (local address 127.10.207.62:43685)
0807 08:37:59.269017 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:59.269019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:59.269051 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:37:59.269134 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:59.269136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:59.269175 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:59.269243 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:59.269650 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.270429 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.271326 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.271653 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.271741 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:59.275350 (+ 3609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:45763: BlockingRecv error: recv got EOF from 127.10.207.1:45763 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":108,"threads_started":1}
Aug 07 08:37:59 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:59 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:59.406683 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:59.400985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:54887)
0807 08:37:59.401218 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:59.401235 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:59.401318 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:59.401498 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:59.401499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:59.401791 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:59.401855 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:59.401858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.402645 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.402647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:59.403296 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:59.403298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.403716 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.403718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:59.403778 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:59.404382 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:59.404389 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:59.406594 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
W20260807 08:37:59.406889 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:59.401176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:54887 (local address 127.10.207.62:43685)
0807 08:37:59.401282 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:59.401284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:59.401294 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:37:59.401405 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:59.401407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:59.401422 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:59.401484 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:59.401937 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.402514 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.403377 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.403596 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.403687 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:59.406812 (+ 3125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:54887: BlockingRecv error: recv got EOF from 127.10.207.2:54887 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 08:37:59 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:37:59 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:37:59.545303 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:37:59.536005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:48133)
0807 08:37:59.536146 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 08:37:59.536157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:37:59.536218 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:37:59.536426 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:37:59.536427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:37:59.536621 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:37:59.536668 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:59.536669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.537344 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.537345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:59.537870 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:37:59.537872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.538213 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.538214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:37:59.538265 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:59.542708 (+ 4443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:37:59.542715 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:37:59.545181 (+ 2466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
W20260807 08:37:59.545353 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:37:59.536228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:48133 (local address 127.10.207.62:43685)
0807 08:37:59.536311 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 08:37:59.536313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:37:59.536319 (+ 6us) server_negotiation.cc:407] Connection header received
0807 08:37:59.536340 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:37:59.536341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:37:59.536354 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:37:59.536416 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:37:59.536755 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.537265 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.537954 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:37:59.538139 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:37:59.538214 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:37:59.545275 (+ 7061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:48133: BlockingRecv error: recv got EOF from 127.10.207.3:48133 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 08:38:00 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:00 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:00.282837 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:00.276276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:49247)
0807 08:38:00.276469 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:00.276486 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:00.276606 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:00.276826 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:00.276827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:00.277042 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:00.277109 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:00.277112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.278203 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.278205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:00.278948 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:00.278950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.279451 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.279453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:00.279508 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:00.280136 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:00.280142 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:00.282741 (+ 2599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
W20260807 08:38:00.282948 24205 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:00.276339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:49247 (local address 127.10.207.62:43685)
0807 08:38:00.276516 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:00.276519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:00.276573 (+ 54us) server_negotiation.cc:407] Connection header received
0807 08:38:00.276704 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:00.276705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:00.276725 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:00.276813 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:00.277216 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.278084 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.279060 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.279356 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.279451 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:00.282863 (+ 3412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:49247: BlockingRecv error: recv got EOF from 127.10.207.1:49247 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
Aug 07 08:38:00 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:00 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:00.412920 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:00.407261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:58499)
0807 08:38:00.407475 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:00.407490 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:00.407589 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:00.407852 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:00.407854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:00.408082 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:00.408165 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:00.408167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.408934 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.408935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:00.409550 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:00.409552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.409942 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.409943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:00.409991 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:00.410646 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:00.410652 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:00.412842 (+ 2190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
W20260807 08:38:00.413053 24205 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:00.407357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:58499 (local address 127.10.207.62:43685)
0807 08:38:00.407481 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:00.407484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:00.407647 (+ 163us) server_negotiation.cc:407] Connection header received
0807 08:38:00.407681 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:00.407682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:00.407695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:00.407766 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:00.408313 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.408833 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.409661 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.409856 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.409927 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:00.412970 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:58499: BlockingRecv error: recv got EOF from 127.10.207.2:58499 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 08:38:00 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:00 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:00.551833 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:00.546336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:51131)
0807 08:38:00.546595 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:00.546609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:00.546732 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:00.546907 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:00.546907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:00.547198 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:00.547264 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:00.547267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.548275 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.548276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:00.548844 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:00.548845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.549189 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.549190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:00.549236 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:00.549744 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:00.549750 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:00.551761 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":97,"threads_started":1}
W20260807 08:38:00.552012 24205 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:00.546502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:51131 (local address 127.10.207.62:43685)
0807 08:38:00.546623 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:00.546626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:00.546745 (+ 119us) server_negotiation.cc:407] Connection header received
0807 08:38:00.546793 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:00.546794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:00.546810 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:00.546926 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:00.547350 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.548172 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.548927 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:00.549110 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:00.549168 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:00.551948 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:51131: BlockingRecv error: recv got EOF from 127.10.207.3:51131 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 08:38:01 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:01 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:01.289419 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:01.283742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:55113)
0807 08:38:01.283976 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:01.283991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:01.284100 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:01.284371 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:01.284372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:01.284583 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:01.284662 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:01.284664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.285413 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.285413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:01.286034 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:01.286036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.286433 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.286434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:01.286484 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:01.287131 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:01.287137 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:01.289296 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":122,"threads_started":1}
W20260807 08:38:01.289448 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:01.283880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:55113 (local address 127.10.207.62:43685)
0807 08:38:01.284025 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:01.284027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:01.284073 (+ 46us) server_negotiation.cc:407] Connection header received
0807 08:38:01.284249 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:01.284250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:01.284271 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:01.284341 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:01.284749 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.285320 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.286148 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.286354 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.286420 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:01.289378 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:55113: BlockingRecv error: recv got EOF from 127.10.207.1:55113 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
Aug 07 08:38:01 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:01 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:01.419101 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:01.413631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:55067)
0807 08:38:01.413822 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:01.413842 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:01.413917 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:01.414072 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:01.414074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:01.414298 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:01.414369 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:01.414371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.415259 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.415261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:01.415875 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:01.415877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.416285 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.416286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:01.416334 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:01.416980 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:01.416985 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:01.419038 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":121,"threads_started":1}
W20260807 08:38:01.419296 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:01.413733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:55067 (local address 127.10.207.62:43685)
0807 08:38:01.413830 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:01.413832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:01.413895 (+ 63us) server_negotiation.cc:407] Connection header received
0807 08:38:01.413970 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:01.413971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:01.413984 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:01.414046 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:01.414460 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.415118 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.415985 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.416197 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.416261 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:01.419185 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:55067: BlockingRecv error: recv got EOF from 127.10.207.2:55067 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 08:38:01 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:01 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:01.558387 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:01.552764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:37247)
0807 08:38:01.552971 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:01.552986 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:01.553088 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:01.553278 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:01.553280 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:01.553512 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:01.553583 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:01.553585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.554295 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.554297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:01.554978 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:01.554980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.555387 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.555388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:01.555462 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:01.556150 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:01.556156 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:01.558312 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":108,"threads_started":1}
W20260807 08:38:01.558619 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:01.552856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:37247 (local address 127.10.207.62:43685)
0807 08:38:01.552943 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:01.552945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:01.553064 (+ 119us) server_negotiation.cc:407] Connection header received
0807 08:38:01.553154 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:01.553156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:01.553171 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:01.553236 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:01.553667 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.554174 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.555089 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:01.555277 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:01.555350 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:01.558492 (+ 3142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:37247: BlockingRecv error: recv got EOF from 127.10.207.3:37247 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 08:38:02 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:02 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:02.295732 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:02.290055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:35883)
0807 08:38:02.290256 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:02.290271 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:02.290348 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:02.290582 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:02.290583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:02.290814 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:02.290898 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:02.290900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.291636 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.291637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:02.292284 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:02.292286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.292652 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.292653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:02.292701 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:02.293327 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:02.293334 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:02.295627 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":88,"threads_started":1}
W20260807 08:38:02.295768 24216 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:02.290163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:35883 (local address 127.10.207.62:43685)
0807 08:38:02.290316 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:02.290318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:02.290332 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:38:02.290419 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:02.290420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:02.290431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:02.290498 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:02.291012 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.291537 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.292369 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.292572 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.292630 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:02.295700 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:35883: BlockingRecv error: recv got EOF from 127.10.207.1:35883 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
Aug 07 08:38:02 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:02 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:02.425865 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:02.419801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:60723)
0807 08:38:02.420058 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:02.420072 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:02.420233 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:02.420426 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:02.420427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:02.420690 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:02.420779 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:02.420783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.421937 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.421938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:02.422522 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:02.422547 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.422956 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.422957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:02.423000 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:02.423543 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:02.423549 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:02.425737 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":145,"threads_started":1}
W20260807 08:38:02.425899 24216 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:02.419888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:60723 (local address 127.10.207.62:43685)
0807 08:38:02.419995 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:02.419999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:02.420201 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:38:02.420320 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:02.420321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:02.420345 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:02.420415 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:02.420911 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.421792 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.422631 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.422854 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.422929 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:02.425826 (+ 2897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:60723: BlockingRecv error: recv got EOF from 127.10.207.2:60723 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 08:38:02 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:02 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:02.564329 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:02.559016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:39917)
0807 08:38:02.559212 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:02.559224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:02.559304 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:02.559471 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:02.559472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:02.559677 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:02.559759 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:02.559761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.560426 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.560427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:02.561002 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:02.561004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.561383 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.561383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:02.561428 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:02.561992 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:02.561997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:02.564258 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":126,"threads_started":1}
W20260807 08:38:02.564442 24216 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:02.559119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:39917 (local address 127.10.207.62:43685)
0807 08:38:02.559199 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:02.559201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:02.559307 (+ 106us) server_negotiation.cc:407] Connection header received
0807 08:38:02.559360 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:02.559361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:02.559376 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:02.559444 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:02.559832 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.560340 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.561105 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:02.561298 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:02.561368 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:02.564359 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:39917: BlockingRecv error: recv got EOF from 127.10.207.3:39917 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 08:38:03 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:03 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:03.301684 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:03.296385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:48077)
0807 08:38:03.296573 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:03.296584 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:03.296641 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:03.296814 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:03.296815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:03.297001 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:03.297056 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:03.297058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.297791 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.297792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:03.298351 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:03.298353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.298756 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.298758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:03.298810 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:03.299348 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:03.299353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:03.301546 (+ 2193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
W20260807 08:38:03.301703 24223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:03.296456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:48077 (local address 127.10.207.62:43685)
0807 08:38:03.296635 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:03.296637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:03.296679 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:38:03.296706 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:03.296706 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:03.296719 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:03.296783 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:03.297139 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.297693 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.298435 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.298656 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.298716 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:03.301627 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:48077: BlockingRecv error: recv got EOF from 127.10.207.1:48077 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
Aug 07 08:38:03 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:03 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:03.432022 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:03.426477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:58435)
0807 08:38:03.426745 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:03.426760 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:03.426859 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:03.427110 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:03.427112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:03.427377 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:03.427485 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:03.427489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.428255 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.428257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:03.428909 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:03.428910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.429275 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.429276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:03.429324 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:03.429816 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:03.429821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:03.431952 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":156,"threads_started":1}
W20260807 08:38:03.432101 24223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:03.426595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:58435 (local address 127.10.207.62:43685)
0807 08:38:03.426710 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:03.426713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:03.426833 (+ 120us) server_negotiation.cc:407] Connection header received
0807 08:38:03.426961 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:03.426962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:03.426978 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:03.427043 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:03.427589 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.428164 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.428997 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.429196 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.429256 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:03.432049 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:58435: BlockingRecv error: recv got EOF from 127.10.207.2:58435 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 08:38:03 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:03 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:03.570876 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:03.565009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:44127)
0807 08:38:03.565284 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:03.565308 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:03.565406 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:03.565636 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:03.565638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:03.565826 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:03.565883 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:03.565884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.566738 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.566740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:03.567355 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:03.567357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.567759 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.567760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:03.567835 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:03.568413 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:03.568419 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:03.570801 (+ 2382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":115,"threads_started":1}
W20260807 08:38:03.571022 24223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:03.565203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:44127 (local address 127.10.207.62:43685)
0807 08:38:03.565321 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:03.565324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:03.565400 (+ 76us) server_negotiation.cc:407] Connection header received
0807 08:38:03.565475 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:03.565477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:03.565496 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:03.565588 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:03.566005 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.566606 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.567446 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:03.567646 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:03.567713 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:03.570922 (+ 3209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:44127: BlockingRecv error: recv got EOF from 127.10.207.3:44127 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 08:38:04 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:04 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:04.307917 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:04.302280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:52265)
0807 08:38:04.302439 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:04.302455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:04.302548 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:04.302793 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:04.302794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:04.302980 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:04.303038 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:04.303039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.304096 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.304097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:04.304676 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:04.304678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.305110 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.305112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:04.305157 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:04.305683 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:04.305687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:04.307844 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
W20260807 08:38:04.308100 24228 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:04.302368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:52265 (local address 127.10.207.62:43685)
0807 08:38:04.302604 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:04.302607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:04.302617 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:38:04.302646 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:04.302647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:04.302661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:04.302764 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:04.303282 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.304004 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.304772 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.304986 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.305106 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:04.307981 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:52265: BlockingRecv error: recv got EOF from 127.10.207.1:52265 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
Aug 07 08:38:04 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:04 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:04.438005 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:04.432679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:41217)
0807 08:38:04.432889 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:04.432901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:04.432983 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:04.433346 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:04.433348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:04.433619 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:04.433703 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:04.433706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.434501 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.434502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:04.435118 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:04.435120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.435504 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.435505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:04.435552 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:04.436080 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:04.436085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:04.437946 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
W20260807 08:38:04.438220 24228 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:04.432820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:41217 (local address 127.10.207.62:43685)
0807 08:38:04.432923 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:04.432926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:04.433095 (+ 169us) server_negotiation.cc:407] Connection header received
0807 08:38:04.433135 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:04.433136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:04.433160 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:04.433262 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:04.433814 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.434404 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.435219 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.435417 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.435488 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:04.438136 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:41217: BlockingRecv error: recv got EOF from 127.10.207.2:41217 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 08:38:04 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:04 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:04.581029 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:04.571573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:55275)
0807 08:38:04.571850 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:04.571863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:04.571947 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:04.572151 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:04.572153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:04.572348 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:04.572407 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:04.572409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.573263 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.573264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:04.573838 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:04.573840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.574235 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.574236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:04.574287 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:04.578650 (+ 4363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:04.578656 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:04.580914 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
W20260807 08:38:04.581084 24228 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:04.571768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:55275 (local address 127.10.207.62:43685)
0807 08:38:04.571860 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:04.571862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:04.571940 (+ 78us) server_negotiation.cc:407] Connection header received
0807 08:38:04.572003 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:04.572003 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:04.572023 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:04.572080 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:04.572506 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.573151 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.573923 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:04.574144 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:04.574203 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:04.581010 (+ 6807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:55275: BlockingRecv error: recv got EOF from 127.10.207.3:55275 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 08:38:05 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:05 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:05.314646 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:05.308528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:42243)
0807 08:38:05.308752 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:05.308768 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:05.308848 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:05.309067 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:05.309068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:05.309287 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:05.309361 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:05.309363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.310066 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.310067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:05.310726 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:05.310728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.311284 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.311285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:05.311334 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:05.311871 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:05.311876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:05.314516 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
W20260807 08:38:05.314710 24233 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:05.308641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:42243 (local address 127.10.207.62:43685)
0807 08:38:05.308811 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:05.308813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:05.308831 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:38:05.308928 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:05.308929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:05.308942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:05.309009 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:05.309449 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.309972 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.310845 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.311188 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.311266 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:05.314631 (+ 3365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:42243: BlockingRecv error: recv got EOF from 127.10.207.1:42243 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
Aug 07 08:38:05 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:05 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:05.444154 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:05.438656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:52321)
0807 08:38:05.438850 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:05.438866 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:05.438947 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:05.439119 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:05.439119 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:05.439305 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:05.439359 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:05.439360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.440163 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.440164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:05.440761 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:05.440763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.441293 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.441295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:05.441365 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:05.441871 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:05.441876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:05.444070 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
W20260807 08:38:05.444306 24233 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:05.438729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:52321 (local address 127.10.207.62:43685)
0807 08:38:05.438800 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:05.438804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:05.439001 (+ 197us) server_negotiation.cc:407] Connection header received
0807 08:38:05.439040 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:05.439041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:05.439055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:05.439099 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:05.439462 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.440044 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.440958 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.441195 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.441278 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:05.444190 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:52321: BlockingRecv error: recv got EOF from 127.10.207.2:52321 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 08:38:05 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:05 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:05.587877 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:05.581708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:52777)
0807 08:38:05.581939 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:05.581952 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:05.582030 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:05.582215 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:05.582216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:05.582439 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:05.582504 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:05.582506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.583247 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.583248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:05.583873 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:05.583874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.584435 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.584435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:05.584485 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:05.585350 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:05.585355 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:05.587786 (+ 2431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":126,"threads_started":1}
W20260807 08:38:05.587966 24233 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:05.581809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:52777 (local address 127.10.207.62:43685)
0807 08:38:05.581887 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:05.581889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:05.582042 (+ 153us) server_negotiation.cc:407] Connection header received
0807 08:38:05.582103 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:05.582104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:05.582122 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:05.582210 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:05.582620 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.583133 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.584005 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:05.584338 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:05.584422 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:05.587911 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:52777: BlockingRecv error: recv got EOF from 127.10.207.3:52777 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 08:38:06 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:06 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:06.320312 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:06.315223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:49781)
0807 08:38:06.315371 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:06.315391 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:06.315440 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:06.315655 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:06.315656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:06.315837 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:06.315893 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:06.315895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.316655 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.316657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:06.317229 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:06.317230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.317699 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.317700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:06.317744 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:06.318310 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:06.318315 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:06.320249 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
W20260807 08:38:06.320547 24237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:06.315307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:49781 (local address 127.10.207.62:43685)
0807 08:38:06.315484 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:06.315486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:06.315495 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:38:06.315529 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:06.315530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:06.315543 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:06.315636 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:06.315971 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.316534 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.317358 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.317607 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.317685 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:06.320399 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:49781: BlockingRecv error: recv got EOF from 127.10.207.1:49781 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
Aug 07 08:38:06 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:06 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:06.450210 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:06.444748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:34189)
0807 08:38:06.445019 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:06.445034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:06.445125 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:06.445290 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:06.445291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:06.445482 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:06.445535 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:06.445537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.446270 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.446271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:06.447203 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:06.447206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.447549 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.447550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:06.447591 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:06.448084 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:06.448089 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:06.450152 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":156,"threads_started":1}
W20260807 08:38:06.450428 24237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:06.444825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:34189 (local address 127.10.207.62:43685)
0807 08:38:06.444914 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:06.444917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:06.445146 (+ 229us) server_negotiation.cc:407] Connection header received
0807 08:38:06.445185 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:06.445186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:06.445200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:06.445261 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:06.445611 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.446189 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.447278 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.447474 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.447535 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:06.450314 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:34189: BlockingRecv error: recv got EOF from 127.10.207.2:34189 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 08:38:06 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:06 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:06.594043 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:06.588588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:45117)
0807 08:38:06.588766 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:06.588777 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:06.588849 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:06.589037 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:06.589038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:06.589253 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:06.589321 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:06.589323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.590007 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.590008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:06.590832 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:06.590835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.591285 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.591286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:06.591334 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:06.591848 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:06.591853 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:06.593962 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
W20260807 08:38:06.594378 24237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:06.588692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:45117 (local address 127.10.207.62:43685)
0807 08:38:06.588752 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:06.588755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:06.588839 (+ 84us) server_negotiation.cc:407] Connection header received
0807 08:38:06.588944 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:06.588944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:06.588957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:06.589010 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:06.589401 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.589903 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.590952 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:06.591214 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:06.591271 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:06.594282 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:45117: BlockingRecv error: recv got EOF from 127.10.207.3:45117 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 08:38:07 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:07 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:07.326212 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:07.320840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:41719)
0807 08:38:07.321011 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:07.321022 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:07.321074 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:07.321291 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:07.321292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:07.321468 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:07.321525 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:07.321526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.322311 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.322312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:07.322912 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:07.322914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.323316 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.323317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:07.323360 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:07.323907 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:07.323912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:07.326094 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
W20260807 08:38:07.326297 24241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:07.320935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:41719 (local address 127.10.207.62:43685)
0807 08:38:07.321114 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:07.321117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:07.321133 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:38:07.321163 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:07.321163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:07.321176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:07.321260 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:07.321615 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.322206 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.323010 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.323239 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.323300 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:07.326218 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:41719: BlockingRecv error: recv got EOF from 127.10.207.1:41719 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
Aug 07 08:38:07 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:07 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:07.456528 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:07.450808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:37273)
0807 08:38:07.450981 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:07.450997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:07.451059 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:07.451258 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:07.451259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:07.451447 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:07.451499 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:07.451500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.452274 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.452275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:07.452847 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:07.452849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.453275 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.453276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:07.453327 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:07.453947 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:07.453952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:07.456401 (+ 2449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
W20260807 08:38:07.456646 24241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:07.450967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:37273 (local address 127.10.207.62:43685)
0807 08:38:07.451037 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:07.451039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:07.451052 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:38:07.451162 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:07.451163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:07.451177 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:07.451227 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:07.451573 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.452165 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.452942 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.453173 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.453247 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:07.456533 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:37273: BlockingRecv error: recv got EOF from 127.10.207.2:37273 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 08:38:07 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:07 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:07.600394 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:07.594690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:35843)
0807 08:38:07.594914 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:07.594930 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:07.595009 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:07.595338 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:07.595339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:07.595562 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:07.595633 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:07.595635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.596362 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.596363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:07.596942 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:07.596943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.597325 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.597326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:07.597370 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:07.597888 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:07.597893 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:07.600321 (+ 2428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
W20260807 08:38:07.600466 24241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:07.594814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:35843 (local address 127.10.207.62:43685)
0807 08:38:07.594997 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:07.594999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:07.595014 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:38:07.595186 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:07.595187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:07.595210 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:07.595302 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:07.595733 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.596262 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.597047 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:07.597236 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:07.597307 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:07.600415 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:35843: BlockingRecv error: recv got EOF from 127.10.207.3:35843 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 08:38:08 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:08 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:08.333055 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:08.326939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:40127)
0807 08:38:08.327110 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:08.327120 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:08.327184 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:08.327388 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:08.327389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:08.327575 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:08.327631 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:08.327633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.328427 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.328429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:08.329032 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:08.329034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.329629 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.329630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:08.329700 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:08.330427 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:08.330432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:08.332990 (+ 2558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
W20260807 08:38:08.333271 24245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:08.327008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:40127 (local address 127.10.207.62:43685)
0807 08:38:08.327221 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:08.327223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:08.327233 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:38:08.327269 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:08.327269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:08.327284 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:08.327360 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:08.327713 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.328302 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.329297 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.329515 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.329587 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:08.333171 (+ 3584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:40127: BlockingRecv error: recv got EOF from 127.10.207.1:40127 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":139,"threads_started":1}
Aug 07 08:38:08 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:08 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:08.462824 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:08.457146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:33185)
0807 08:38:08.457398 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:08.457415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:08.457536 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:08.457764 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:08.457766 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:08.458002 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:08.458066 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:08.458067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.458888 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.458889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:08.459466 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:08.459468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.459842 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.459843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:08.459899 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:08.460500 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:08.460506 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:08.462710 (+ 2204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
W20260807 08:38:08.462873 24245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:08.457269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:33185 (local address 127.10.207.62:43685)
0807 08:38:08.457385 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:08.457388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:08.457507 (+ 119us) server_negotiation.cc:407] Connection header received
0807 08:38:08.457653 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:08.457654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:08.457676 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:08.457752 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:08.458155 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.458790 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.459559 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.459762 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.459844 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:08.462801 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:33185: BlockingRecv error: recv got EOF from 127.10.207.2:33185 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 08:38:08 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:08 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.3@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:08.606644 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:08.601084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.3:48677)
0807 08:38:08.601347 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:08.601362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:08.601442 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:08.601608 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:08.601609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:08.601795 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:08.601849 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:08.601851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.602662 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.602663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:08.603454 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:08.603457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.603834 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.603835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:08.603877 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:08.604459 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:08.604463 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:08.606499 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":153,"threads_started":1}
W20260807 08:38:08.606912 24245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:08.601143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:48677 (local address 127.10.207.62:43685)
0807 08:38:08.601238 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:08.601242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:08.601442 (+ 200us) server_negotiation.cc:407] Connection header received
0807 08:38:08.601503 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:08.601503 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:08.601521 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:08.601580 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:08.601927 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.602568 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.603556 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:08.603758 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:08.603822 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:08.606680 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.3:48677: BlockingRecv error: recv got EOF from 127.10.207.3:48677 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 08:38:09 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:09 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.1@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:09.339604 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:09.333747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.1:39655)
0807 08:38:09.333955 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:09.333973 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:09.334040 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:09.334252 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:09.334253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:09.334466 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:09.334581 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:09.334583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.335266 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.335267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:09.335857 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:09.335859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.336303 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.336304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:09.336362 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:09.337033 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:09.337039 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:09.339507 (+ 2468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":129,"threads_started":1}
W20260807 08:38:09.339684 24251 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:09.333917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:39655 (local address 127.10.207.62:43685)
0807 08:38:09.334084 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:09.334087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:09.334098 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:38:09.334133 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:09.334134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:09.334146 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:09.334220 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:09.334670 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.335181 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.335970 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.336186 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.336269 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:09.339626 (+ 3357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.1:39655: BlockingRecv error: recv got EOF from 127.10.207.1:39655 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
Aug 07 08:38:09 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:38:09 dist-test-slave-h6n0 krb5kdc[22751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.207.2@KRBTEST.COM for oryx/127.10.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:38:09.469116 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:09.463505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43685 (local address 127.10.207.2:46969)
0807 08:38:09.463690 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:09.463703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:09.463785 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:09.464088 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:09.464089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:09.464310 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:09.464385 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:09.464388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.465301 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.465302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:09.465893 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:09.465894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.466446 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.466447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:09.466596 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:09.467180 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:09.467184 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:09.469047 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.207.62:43685: Server oryx/127.10.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
W20260807 08:38:09.469322 24251 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:09.463603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:46969 (local address 127.10.207.62:43685)
0807 08:38:09.463718 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:09.463721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:09.463790 (+ 69us) server_negotiation.cc:407] Connection header received
0807 08:38:09.463850 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:09.463851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:09.463865 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:09.463927 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:09.464588 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.465149 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.466017 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.466319 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.466393 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:09.469235 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.207.2:46969: BlockingRecv error: recv got EOF from 127.10.207.2:46969 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:38:09.502410 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 23599
I20260807 08:38:09.516848 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 23812
I20260807 08:38:09.524312 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 23983
I20260807 08:38:09.535243 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 23499
2026-08-07T08:38:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17908 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T08:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:09Z Disabled control of system clock
I20260807 08:38:09.574120 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:40767
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:40767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:38:09.661907 24258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:09.662110 24258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:09.662132 24258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:09.663338 24258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:09.663386 24258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:09.663400 24258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:09.663412 24258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:09.663425 24258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:09.664776 24258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:40767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:09.665059 24258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:09.665364 24258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:09.671001 24267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:09.670974 24270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:09.671005 24258 server_base.cc:1034] running on GCE node
W20260807 08:38:09.670974 24265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:09.671633 24258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:09.671932 24258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:09.673105 24258 hybrid_clock.cc:648] HybridClock initialized: now 1786091889673097 us; error 41 us; skew 500 ppm
I20260807 08:38:09.677673 24258 webserver.cc:460] Webserver started at http://127.10.207.62:42347/ using document root <none> and password file <none>
I20260807 08:38:09.677963 24258 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:09.678018 24258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:09.678203 24258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:09.679335 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "2c7ec94aceb64fc4a947c202241a36ad"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "ed41b0075281f7aa764e66592095c851"
server_key_iv: "cc68140202057e86ba5c9343638beccb"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.679698 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "2c7ec94aceb64fc4a947c202241a36ad"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "ed41b0075281f7aa764e66592095c851"
server_key_iv: "cc68140202057e86ba5c9343638beccb"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.681105 24258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:38:09.681982 24279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:09.682220 24258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:38:09.682349 24258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "2c7ec94aceb64fc4a947c202241a36ad"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "ed41b0075281f7aa764e66592095c851"
server_key_iv: "cc68140202057e86ba5c9343638beccb"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.682435 24258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:09.690924 24258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:09.691196 24258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:09.691314 24258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:09.695906 24258 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:40767
I20260807 08:38:09.696022 24357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:40767 every 8 connection(s)
I20260807 08:38:09.696413 24258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:09.697242 24358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:09.699791 24358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad: Bootstrap starting.
I20260807 08:38:09.700327 24358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:09.700663 24358 log.cc:824] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:09.700739 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 24258
I20260807 08:38:09.700829 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:38:09.701068 11068 external_mini_cluster.cc:1428] Setting key c76b9a2d78abdd805c644c730abfe27b
I20260807 08:38:09.701527 24358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad: No bootstrap required, opened a new log
I20260807 08:38:09.703665 24358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7ec94aceb64fc4a947c202241a36ad" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40767 } }
I20260807 08:38:09.703822 24358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:09.703873 24358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7ec94aceb64fc4a947c202241a36ad, State: Initialized, Role: FOLLOWER
I20260807 08:38:09.704025 24358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7ec94aceb64fc4a947c202241a36ad" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40767 } }
I20260807 08:38:09.704134 24358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:09.704180 24358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:09.704231 24358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:09.704758 24358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7ec94aceb64fc4a947c202241a36ad" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40767 } }
I20260807 08:38:09.704876 24358 leader_election.cc:302] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c7ec94aceb64fc4a947c202241a36ad; no voters:
I20260807 08:38:09.705083 24358 leader_election.cc:288] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:38:09.705117 24365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:09.705243 24365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [term 1 LEADER]: Becoming Leader. State: Replica: 2c7ec94aceb64fc4a947c202241a36ad, State: Running, Role: LEADER
I20260807 08:38:09.705361 24358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:09.705438 24365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7ec94aceb64fc4a947c202241a36ad" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40767 } }
I20260807 08:38:09.705981 24369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c7ec94aceb64fc4a947c202241a36ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7ec94aceb64fc4a947c202241a36ad" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40767 } } }
I20260807 08:38:09.705999 24366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c7ec94aceb64fc4a947c202241a36ad. Latest consensus state: current_term: 1 leader_uuid: "2c7ec94aceb64fc4a947c202241a36ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7ec94aceb64fc4a947c202241a36ad" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40767 } } }
I20260807 08:38:09.706102 24369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:09.706135 24366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:09.706967 24375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:09.707589 24375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:09.708889 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:09.701760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.10.207.62:40767)
0807 08:38:09.702007 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:09.702018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:09.702057 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:38:09.702936 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:09.702944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:09.703010 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:09.703170 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:09.703420 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.705227 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.706344 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.707023 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.707537 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:09.707959 (+ 422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:09.707964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:09.707965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:09.708223 (+ 258us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:09.708278 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:09.708295 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:09.708367 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:09.708533 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:38:09.708623 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260807 08:38:09.710031 24375 catalog_manager.cc:1347] Generated new cluster ID: a0103d310bda45fda542e0e7c5cd282e
I20260807 08:38:09.710088 24375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:09.715618 24375 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:38:09.716355 24375 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:09.720594 24375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c7ec94aceb64fc4a947c202241a36ad: Generated new TSK 0
I20260807 08:38:09.720757 24375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:38:09.722142 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40767
--builtin_ntp_servers=127.10.207.20:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 08:38:09.815286 24395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:09.815560 24395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:09.815598 24395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:09.816820 24395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:09.816887 24395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:09.816932 24395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:38:09.818415 24395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:09.818773 24395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:09.819147 24395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:09.825002 24401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:09.825139 24402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:09.825002 24404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:09.825758 24395 server_base.cc:1034] running on GCE node
I20260807 08:38:09.826004 24395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:09.826293 24395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:09.827452 24395 hybrid_clock.cc:648] HybridClock initialized: now 1786091889827430 us; error 45 us; skew 500 ppm
I20260807 08:38:09.828926 24395 webserver.cc:460] Webserver started at http://127.10.207.1:34635/ using document root <none> and password file <none>
I20260807 08:38:09.829228 24395 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:09.829305 24395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:09.829473 24395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:09.830862 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "486cc03999bd414fa9a457e74dedca81"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "bf21eb66098626c898e9397a5f6379f2"
server_key_iv: "924914e69fdad742b5f7ec4a5052775f"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.831339 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "486cc03999bd414fa9a457e74dedca81"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "bf21eb66098626c898e9397a5f6379f2"
server_key_iv: "924914e69fdad742b5f7ec4a5052775f"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.833035 24395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:38:09.833871 24412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:09.834088 24395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:09.834213 24395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "486cc03999bd414fa9a457e74dedca81"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "bf21eb66098626c898e9397a5f6379f2"
server_key_iv: "924914e69fdad742b5f7ec4a5052775f"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.834300 24395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:09.846258 24395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:09.846685 24395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:09.846853 24395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:09.847122 24395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:09.847525 24395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:09.847564 24395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:09.847589 24395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:09.847625 24395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:09.855540 24395 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:42001
I20260807 08:38:09.855819 24581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:42001 every 8 connection(s)
I20260807 08:38:09.856272 24395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:09.859095 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 24395
I20260807 08:38:09.859218 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:38:09.859416 11068 external_mini_cluster.cc:1428] Setting key 950bc14c23ac0ce2b2c31350754953d8
I20260807 08:38:09.860673 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40767
--builtin_ntp_servers=127.10.207.20:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 08:38:09.863173 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:09.857239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:54311 (local address 127.10.207.62:40767)
0807 08:38:09.857311 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:09.857314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:09.857568 (+ 254us) server_negotiation.cc:407] Connection header received
0807 08:38:09.858117 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:09.858119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:09.858136 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:09.858229 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:09.859060 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.859798 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.861307 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.861628 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.861718 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:09.862795 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:09.862798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:09.862798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:09.862805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:09.862836 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:09.862842 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:09.862925 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:09.863048 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 08:38:09.863089 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:38:09.863528 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:09.857162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:40767 (local address 127.10.207.1:54311)
0807 08:38:09.857400 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:09.857442 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:09.857987 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:09.858318 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:09.858322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:09.858330 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:09.858929 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:09.858936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.859932 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.859934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:09.861197 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:09.861202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:09.861741 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:09.861743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:09.861874 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:09.862621 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:09.862622 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:09.862652 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:09.862675 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:09.862928 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:09.862935 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:09.862939 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:38:09.863034 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:38:09.863144 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":121,"threads_started":1}
I20260807 08:38:09.864065 24582 heartbeater.cc:347] Connected to a master server at 127.10.207.62:40767
I20260807 08:38:09.864195 24582 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:09.864452 24582 heartbeater.cc:511] Master 127.10.207.62:40767 requested a full tablet report, sending...
I20260807 08:38:09.865195 24306 ts_manager.cc:194] Registered new tserver with Master: 486cc03999bd414fa9a457e74dedca81 (127.10.207.1:42001)
I20260807 08:38:09.865862 24306 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.1:54311
W20260807 08:38:09.954916 24585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:09.955142 24585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:09.955163 24585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:09.956357 24585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:09.956400 24585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:09.956425 24585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:38:09.957957 24585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:09.958232 24585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:09.958623 24585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:09.964160 24592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:09.964140 24591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:09.964143 24594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:09.964545 24585 server_base.cc:1034] running on GCE node
I20260807 08:38:09.964784 24585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:09.965056 24585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:09.966250 24585 hybrid_clock.cc:648] HybridClock initialized: now 1786091889966234 us; error 44 us; skew 500 ppm
I20260807 08:38:09.968148 24585 webserver.cc:460] Webserver started at http://127.10.207.2:40957/ using document root <none> and password file <none>
I20260807 08:38:09.968461 24585 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:09.968600 24585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:09.968763 24585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:09.970412 24585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "a5df0b69ce7e4c47923c4f24c4232b76"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "84db930c2e8b2948382c6cabb71855a4"
server_key_iv: "15032a8522da19ca21ead998cb36d78a"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.970863 24585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "a5df0b69ce7e4c47923c4f24c4232b76"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "84db930c2e8b2948382c6cabb71855a4"
server_key_iv: "15032a8522da19ca21ead998cb36d78a"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.972592 24585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:09.973464 24602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:09.973664 24585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:09.973778 24585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "a5df0b69ce7e4c47923c4f24c4232b76"
format_stamp: "Formatted at 2026-08-07 08:38:09 on dist-test-slave-h6n0"
server_key: "84db930c2e8b2948382c6cabb71855a4"
server_key_iv: "15032a8522da19ca21ead998cb36d78a"
server_key_version: "encryptionkey@0"
I20260807 08:38:09.973866 24585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:09.992720 24585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:09.993062 24585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:09.993225 24585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:09.993510 24585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:09.993898 24585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:09.993997 24585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:09.994050 24585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:09.994076 24585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.001667 24585 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:40585
I20260807 08:38:10.001760 24758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:40585 every 8 connection(s)
I20260807 08:38:10.002204 24585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:38:10.008078 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.003116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:45175 (local address 127.10.207.62:40767)
0807 08:38:10.003178 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.003180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.003435 (+ 255us) server_negotiation.cc:407] Connection header received
0807 08:38:10.004011 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.004013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.004025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.004081 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.004785 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.005305 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.006500 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.006812 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.006891 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:10.007781 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:10.007782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:10.007783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:10.007787 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:10.007801 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:10.007804 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:10.007849 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:10.007964 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:38:10.007988 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:38:10.008303 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.003086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:40767 (local address 127.10.207.2:45175)
0807 08:38:10.003304 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:10.003339 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:10.003892 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:10.004153 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:10.004157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:10.004165 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:10.004693 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:10.004698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.005407 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.005408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:10.006407 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:10.006409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.006900 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.006901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:10.007057 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:10.007627 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:10.007628 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:10.007656 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:10.007679 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:10.007849 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:10.007855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:10.007858 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:38:10.007960 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 08:38:10.008075 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260807 08:38:10.008899 24761 heartbeater.cc:347] Connected to a master server at 127.10.207.62:40767
I20260807 08:38:10.009042 24761 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:10.009125 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 24585
I20260807 08:38:10.009253 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:38:10.009320 24761 heartbeater.cc:511] Master 127.10.207.62:40767 requested a full tablet report, sending...
I20260807 08:38:10.009470 11068 external_mini_cluster.cc:1428] Setting key aef1b92604a10362120646819d327f8e
I20260807 08:38:10.009855 24306 ts_manager.cc:194] Registered new tserver with Master: a5df0b69ce7e4c47923c4f24c4232b76 (127.10.207.2:40585)
I20260807 08:38:10.010452 24306 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.2:45175
I20260807 08:38:10.010885 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40767
--builtin_ntp_servers=127.10.207.20:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 08:38:10.101852 24765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:10.102075 24765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:10.102098 24765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:10.103299 24765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:10.103350 24765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:10.103377 24765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:38:10.104719 24765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:10.105020 24765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:10.105376 24765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:10.111011 24774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.111020 24773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:10.111042 24765 server_base.cc:1034] running on GCE node
W20260807 08:38:10.111030 24778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:10.111538 24765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:10.111824 24765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:10.113024 24765 hybrid_clock.cc:648] HybridClock initialized: now 1786091890113011 us; error 51 us; skew 500 ppm
I20260807 08:38:10.114507 24765 webserver.cc:460] Webserver started at http://127.10.207.3:41853/ using document root <none> and password file <none>
I20260807 08:38:10.114804 24765 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:10.114879 24765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:10.115025 24765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:10.116044 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "f2b4ac8035d1471c9acee88055024bd6"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "13a71b53ee3764820c26820417677aed"
server_key_iv: "cc5a16d2ce49946a4e8ae213fb100f57"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.116410 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "f2b4ac8035d1471c9acee88055024bd6"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "13a71b53ee3764820c26820417677aed"
server_key_iv: "cc5a16d2ce49946a4e8ae213fb100f57"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.117910 24765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:38:10.118880 24784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.119220 24765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:38:10.119380 24765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "f2b4ac8035d1471c9acee88055024bd6"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "13a71b53ee3764820c26820417677aed"
server_key_iv: "cc5a16d2ce49946a4e8ae213fb100f57"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.119491 24765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:10.133656 24765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:10.133970 24765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:10.134131 24765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:10.134380 24765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:10.134837 24765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:10.134900 24765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.134938 24765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:10.134964 24765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.143586 24765 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:38297
I20260807 08:38:10.143671 24941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:38297 every 8 connection(s)
I20260807 08:38:10.144129 24765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:10.148053 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 24765
I20260807 08:38:10.148165 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:38:10.148370 11068 external_mini_cluster.cc:1428] Setting key 398d3179c41d4ea8260ca82e3d4d50c7
I20260807 08:38:10.150337 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.145142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:33799 (local address 127.10.207.62:40767)
0807 08:38:10.145234 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.145238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.145531 (+ 293us) server_negotiation.cc:407] Connection header received
0807 08:38:10.146133 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.146134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.146146 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.146216 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.147015 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.147501 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.148778 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.149089 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.149168 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:10.150121 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:10.150122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:10.150122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:10.150126 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:10.150141 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:10.150145 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:10.150194 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:10.150259 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:38:10.150281 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:38:10.150669 24946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.145102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:40767 (local address 127.10.207.3:33799)
0807 08:38:10.145365 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:10.145408 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:10.146011 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:10.146306 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:10.146310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:10.146320 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:10.146933 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:10.146938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.147590 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.147591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:10.148660 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:10.148664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:10.149190 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:10.149192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:10.149356 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:10.149974 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:10.149975 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:10.150008 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:10.150034 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:10.150197 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:10.150199 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:10.150202 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:38:10.150254 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:38:10.150343 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":126,"threads_started":1}
I20260807 08:38:10.151281 24943 heartbeater.cc:347] Connected to a master server at 127.10.207.62:40767
I20260807 08:38:10.151398 24943 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:10.151603 24943 heartbeater.cc:511] Master 127.10.207.62:40767 requested a full tablet report, sending...
I20260807 08:38:10.152076 24306 ts_manager.cc:194] Registered new tserver with Master: f2b4ac8035d1471c9acee88055024bd6 (127.10.207.3:38297)
I20260807 08:38:10.152477 24306 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.3:33799
I20260807 08:38:10.160600 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:38:10.162580 24364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:10.161989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.10.207.62:40767)
0807 08:38:10.162055 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.162057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.162210 (+ 153us) server_negotiation.cc:407] Connection header received
0807 08:38:10.162265 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.162265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.162273 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.162317 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.162487 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53996: BlockingRecv error: recv got EOF from 127.0.0.1:53996 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:38:10.162706 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 24395
I20260807 08:38:10.194671 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 24585
I20260807 08:38:10.205895 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 24765
I20260807 08:38:10.233676 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 24258
2026-08-07T08:38:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (695 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T08:38:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:10Z Disabled control of system clock
I20260807 08:38:10.270267 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44081
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:44081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:38:10.366210 24963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:10.366567 24963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:10.366605 24963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:10.368721 24963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:10.368793 24963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:10.368818 24963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:10.368922 24963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:10.368979 24963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:10.371332 24963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:44081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:10.371632 24963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:10.372143 24963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:10.380893 24968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.380899 24969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.381202 24974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:10.382678 24963 server_base.cc:1034] running on GCE node
I20260807 08:38:10.383006 24963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:10.383414 24963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:10.384589 24963 hybrid_clock.cc:648] HybridClock initialized: now 1786091890384571 us; error 29 us; skew 500 ppm
I20260807 08:38:10.386166 24963 webserver.cc:460] Webserver started at http://127.10.207.62:42011/ using document root <none> and password file <none>
I20260807 08:38:10.386482 24963 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:10.386596 24963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:10.386766 24963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:10.387907 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "16e22295f41249f5bcf818c3237831ef"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "10e64c6ce74759ecba415c06afaab0f1"
server_key_iv: "42721ec065e5d3c3a4e579694c0fc041"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.388314 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "16e22295f41249f5bcf818c3237831ef"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "10e64c6ce74759ecba415c06afaab0f1"
server_key_iv: "42721ec065e5d3c3a4e579694c0fc041"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.389978 24963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:38:10.390878 24987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.391094 24963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:10.391206 24963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "16e22295f41249f5bcf818c3237831ef"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "10e64c6ce74759ecba415c06afaab0f1"
server_key_iv: "42721ec065e5d3c3a4e579694c0fc041"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.391294 24963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:10.394647 24963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:10.394919 24963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:10.395066 24963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:10.399559 24963 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:44081
I20260807 08:38:10.399695 25061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:44081 every 8 connection(s)
I20260807 08:38:10.399978 24963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:10.400879 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:10.404265 25062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef: Bootstrap starting.
I20260807 08:38:10.404943 25062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:10.405242 25062 log.cc:824] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:10.406034 25062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef: No bootstrap required, opened a new log
I20260807 08:38:10.407162 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 24963
I20260807 08:38:10.407287 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:38:10.407503 11068 external_mini_cluster.cc:1428] Setting key 3acc6646cd6d73c6906b762c85809adb
I20260807 08:38:10.408206 25062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e22295f41249f5bcf818c3237831ef" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44081 } }
I20260807 08:38:10.408438 25062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:10.408481 25062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e22295f41249f5bcf818c3237831ef, State: Initialized, Role: FOLLOWER
I20260807 08:38:10.408613 25062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e22295f41249f5bcf818c3237831ef" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44081 } }
I20260807 08:38:10.408699 25062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:10.408739 25062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:10.408794 25062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:10.409643 25062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e22295f41249f5bcf818c3237831ef" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44081 } }
I20260807 08:38:10.409783 25062 leader_election.cc:302] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16e22295f41249f5bcf818c3237831ef; no voters:
I20260807 08:38:10.409994 25062 leader_election.cc:288] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:38:10.410082 25069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:10.410283 25069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [term 1 LEADER]: Becoming Leader. State: Replica: 16e22295f41249f5bcf818c3237831ef, State: Running, Role: LEADER
I20260807 08:38:10.410318 25062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:10.410454 25069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e22295f41249f5bcf818c3237831ef" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44081 } }
I20260807 08:38:10.410895 25069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16e22295f41249f5bcf818c3237831ef. Latest consensus state: current_term: 1 leader_uuid: "16e22295f41249f5bcf818c3237831ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e22295f41249f5bcf818c3237831ef" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44081 } } }
I20260807 08:38:10.411002 25069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:10.410908 25071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16e22295f41249f5bcf818c3237831ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e22295f41249f5bcf818c3237831ef" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44081 } } }
I20260807 08:38:10.411175 25071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:10.411298 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.408123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.10.207.62:44081)
0807 08:38:10.408393 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.408404 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.408444 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:38:10.408578 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.408586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.408637 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.408846 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.410055 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:10.410060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:10.410061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:10.410066 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:10.410124 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:10.410142 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:10.410224 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:10.410501 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 08:38:10.410602 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
I20260807 08:38:10.411314 25079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:10.412001 25079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:10.414065 25079 catalog_manager.cc:1347] Generated new cluster ID: cdd729c7bc3e4de3b2fbf61b2693026d
I20260807 08:38:10.414126 25079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:38:10.432307 25079 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:38:10.432525 25079 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:10.437237 25079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16e22295f41249f5bcf818c3237831ef: Generated new TSK 0
I20260807 08:38:10.437502 25079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:38:10.447031 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44081
--builtin_ntp_servers=127.10.207.20:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:38:10.541699 25097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:10.541926 25097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:10.541950 25097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:10.543255 25097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:10.543301 25097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:10.543329 25097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:38:10.544703 25097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:10.544968 25097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:10.545321 25097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:10.551049 25106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.550901 25103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.550894 25104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:10.551189 25097 server_base.cc:1034] running on GCE node
I20260807 08:38:10.551479 25097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:10.551719 25097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:10.552999 25097 hybrid_clock.cc:648] HybridClock initialized: now 1786091890552870 us; error 60 us; skew 500 ppm
I20260807 08:38:10.554700 25097 webserver.cc:460] Webserver started at http://127.10.207.1:33025/ using document root <none> and password file <none>
I20260807 08:38:10.555084 25097 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:10.555161 25097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:10.555315 25097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:10.556608 25097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "c85e8f1e75294018924ffa11ea7ebcef"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "1acbf1eb0745a5abd44d830c12c5e65e"
server_key_iv: "e1ac924c6836e9f2c3e426625ab1fbf1"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.557186 25097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "c85e8f1e75294018924ffa11ea7ebcef"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "1acbf1eb0745a5abd44d830c12c5e65e"
server_key_iv: "e1ac924c6836e9f2c3e426625ab1fbf1"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.558962 25097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260807 08:38:10.559945 25114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.560156 25097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:10.560256 25097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "c85e8f1e75294018924ffa11ea7ebcef"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "1acbf1eb0745a5abd44d830c12c5e65e"
server_key_iv: "e1ac924c6836e9f2c3e426625ab1fbf1"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.560338 25097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:10.563685 25097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:10.563985 25097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:10.564246 25097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:10.564508 25097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:10.564833 25097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:10.564893 25097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.564947 25097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:10.564977 25097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.572445 25097 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:45959
I20260807 08:38:10.572551 25290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:45959 every 8 connection(s)
I20260807 08:38:10.573056 25097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:10.573277 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 25097
I20260807 08:38:10.573377 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:38:10.573633 11068 external_mini_cluster.cc:1428] Setting key 30e1dbc12d6f8f81fe67a92638efcc74
I20260807 08:38:10.574882 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44081
--builtin_ntp_servers=127.10.207.20:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 08:38:10.577332 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.574377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:37647 (local address 127.10.207.62:44081)
0807 08:38:10.574436 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.574438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.574523 (+ 85us) server_negotiation.cc:407] Connection header received
0807 08:38:10.575376 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.575378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.575389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.575479 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.576854 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:10.576855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:10.576856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:10.576862 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:10.576887 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:10.576894 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:10.577010 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:10.577209 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 08:38:10.577233 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:38:10.577572 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.574123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44081 (local address 127.10.207.1:37647)
0807 08:38:10.574359 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:10.574410 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:10.575221 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:10.575541 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:10.575544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:10.575554 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:10.576649 (+ 1095us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:10.576650 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:10.576683 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:10.576715 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:10.577039 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:10.577046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:10.577050 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:38:10.577204 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 08:38:10.577293 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":123,"threads_started":1}
I20260807 08:38:10.578320 25291 heartbeater.cc:347] Connected to a master server at 127.10.207.62:44081
I20260807 08:38:10.578464 25291 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:10.578673 25291 heartbeater.cc:511] Master 127.10.207.62:44081 requested a full tablet report, sending...
I20260807 08:38:10.579452 25009 ts_manager.cc:194] Registered new tserver with Master: c85e8f1e75294018924ffa11ea7ebcef (127.10.207.1:45959)
W20260807 08:38:10.662168 25295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:10.662384 25295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:10.662408 25295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:10.663627 25295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:10.663677 25295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:10.663707 25295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:38:10.665066 25295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:10.665268 25295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:10.665525 25295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:10.670885 25303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.670853 25306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.670877 25308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:10.671314 25295 server_base.cc:1034] running on GCE node
I20260807 08:38:10.671550 25295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:10.671757 25295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:10.672915 25295 hybrid_clock.cc:648] HybridClock initialized: now 1786091890672893 us; error 47 us; skew 500 ppm
I20260807 08:38:10.674222 25295 webserver.cc:460] Webserver started at http://127.10.207.2:42781/ using document root <none> and password file <none>
I20260807 08:38:10.674502 25295 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:10.674613 25295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:10.674750 25295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:10.675782 25295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "2ebc6deb385e41ba9f2b3893d2af2f73"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "8be759772331019c532b1682716e9821"
server_key_iv: "17442c85a38183704b07c8bf05aae115"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.676153 25295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "2ebc6deb385e41ba9f2b3893d2af2f73"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "8be759772331019c532b1682716e9821"
server_key_iv: "17442c85a38183704b07c8bf05aae115"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.677629 25295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:38:10.678608 25321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.678951 25295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:10.679090 25295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "2ebc6deb385e41ba9f2b3893d2af2f73"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "8be759772331019c532b1682716e9821"
server_key_iv: "17442c85a38183704b07c8bf05aae115"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.679185 25295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:10.682749 25295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:10.683033 25295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:10.683190 25295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:10.683420 25295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:10.683761 25295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:10.683822 25295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.683859 25295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:10.683889 25295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.692082 25295 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:46419
I20260807 08:38:10.692126 25486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:46419 every 8 connection(s)
I20260807 08:38:10.692602 25295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:38:10.696414 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.693549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:49463 (local address 127.10.207.62:44081)
0807 08:38:10.693665 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.693668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.693908 (+ 240us) server_negotiation.cc:407] Connection header received
0807 08:38:10.694656 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.694657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.694666 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.694733 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.696076 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:10.696078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:10.696078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:10.696083 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:10.696139 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:10.696148 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:10.696222 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:10.696306 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 08:38:10.696325 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:38:10.696636 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.693492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44081 (local address 127.10.207.2:49463)
0807 08:38:10.693755 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:10.693796 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:10.694466 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:10.694814 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:10.694819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:10.694830 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:10.695859 (+ 1029us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:10.695859 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:10.695893 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:10.695926 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:10.696220 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:10.696228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:10.696232 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:38:10.696310 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:38:10.696384 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
I20260807 08:38:10.697232 25488 heartbeater.cc:347] Connected to a master server at 127.10.207.62:44081
I20260807 08:38:10.697345 25488 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:10.697502 25488 heartbeater.cc:511] Master 127.10.207.62:44081 requested a full tablet report, sending...
I20260807 08:38:10.697928 25009 ts_manager.cc:194] Registered new tserver with Master: 2ebc6deb385e41ba9f2b3893d2af2f73 (127.10.207.2:46419)
I20260807 08:38:10.702786 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 25295
I20260807 08:38:10.702920 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:38:10.703153 11068 external_mini_cluster.cc:1428] Setting key a1cd735d091b2bb679013ca85b44b20b
I20260807 08:38:10.704523 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44081
--builtin_ntp_servers=127.10.207.20:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:38:10.800419 25493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:10.800688 25493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:10.800714 25493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:10.802083 25493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:10.802147 25493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:10.802178 25493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:38:10.803720 25493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:44081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:10.803997 25493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:10.804347 25493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:10.810199 25512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:10.810220 25508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:10.810371 25493 server_base.cc:1034] running on GCE node
W20260807 08:38:10.810410 25505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:10.810840 25493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:10.811203 25493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:10.812413 25493 hybrid_clock.cc:648] HybridClock initialized: now 1786091890812344 us; error 89 us; skew 500 ppm
I20260807 08:38:10.813943 25493 webserver.cc:460] Webserver started at http://127.10.207.3:33641/ using document root <none> and password file <none>
I20260807 08:38:10.814296 25493 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:10.814378 25493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:10.814606 25493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:10.815997 25493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "02953fae1d5d46d0baa659ef061b1a03"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "df8999937019ee723e116ed7ea52598d"
server_key_iv: "47b9a153588538757057ac22263641de"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.816479 25493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "02953fae1d5d46d0baa659ef061b1a03"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "df8999937019ee723e116ed7ea52598d"
server_key_iv: "47b9a153588538757057ac22263641de"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.818087 25493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:38:10.819084 25519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.819320 25493 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:38:10.819437 25493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "02953fae1d5d46d0baa659ef061b1a03"
format_stamp: "Formatted at 2026-08-07 08:38:10 on dist-test-slave-h6n0"
server_key: "df8999937019ee723e116ed7ea52598d"
server_key_iv: "47b9a153588538757057ac22263641de"
server_key_version: "encryptionkey@0"
I20260807 08:38:10.819531 25493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:10.823426 25493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:10.823732 25493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:10.823925 25493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:10.824162 25493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:10.824520 25493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:10.824558 25493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.824579 25493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:10.824630 25493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:10.831939 25493 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:36661
I20260807 08:38:10.831995 25685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:36661 every 8 connection(s)
I20260807 08:38:10.832366 25493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:10.835829 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.833261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:52279 (local address 127.10.207.62:44081)
0807 08:38:10.833313 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.833315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.833610 (+ 295us) server_negotiation.cc:407] Connection header received
0807 08:38:10.834186 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.834187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.834196 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.834268 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.835542 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:10.835543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:10.835544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:38:10.835548 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:10.835564 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:38:10.835568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:10.835628 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:10.835746 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:38:10.835764 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:38:10.835988 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:10.833231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44081 (local address 127.10.207.3:52279)
0807 08:38:10.833433 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:10.833472 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:10.834029 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:10.834331 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:10.834334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:10.834343 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:10.835374 (+ 1031us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:38:10.835375 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:10.835407 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:38:10.835438 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:10.835629 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:10.835632 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:10.835634 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:38:10.835699 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:38:10.835754 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260807 08:38:10.836633 25690 heartbeater.cc:347] Connected to a master server at 127.10.207.62:44081
I20260807 08:38:10.836733 25690 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:10.836866 25690 heartbeater.cc:511] Master 127.10.207.62:44081 requested a full tablet report, sending...
I20260807 08:38:10.837288 25009 ts_manager.cc:194] Registered new tserver with Master: 02953fae1d5d46d0baa659ef061b1a03 (127.10.207.3:36661)
I20260807 08:38:10.841758 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 25493
I20260807 08:38:10.841888 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:38:10.842126 11068 external_mini_cluster.cc:1428] Setting key f5a3b3b95a33c458143b44fdc07873a7
I20260807 08:38:10.843290 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:38:10.845602 25066 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:10.845057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.10.207.62:44081)
0807 08:38:10.845160 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:10.845163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:10.845268 (+ 105us) server_negotiation.cc:407] Connection header received
0807 08:38:10.845339 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:10.845340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:10.845345 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:10.845403 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:10.845541 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44542: BlockingRecv error: recv got EOF from 127.0.0.1:44542 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:38:10.845832 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 25097
I20260807 08:38:10.859403 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 25295
I20260807 08:38:10.880846 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 25493
I20260807 08:38:10.897198 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 24963
2026-08-07T08:38:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (667 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:38:10 dist-test-slave-h6n0 krb5kdc[25706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:38:10 dist-test-slave-h6n0 krb5kdc[25706](info): set up 2 sockets
Aug 07 08:38:10 dist-test-slave-h6n0 krb5kdc[25706](info): commencing operation
krb5kdc: starting...
W20260807 08:38:12.224229 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:38:12 dist-test-slave-h6n0 krb5kdc[25706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:38:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:38:12.361961 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:37937
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:12.455456 25721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:12.455693 25721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:12.455717 25721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:12.456928 25721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:38:12.456974 25721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:12.456990 25721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:12.457002 25721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:38:12.457015 25721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:38:12.458657 25721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:12.458942 25721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:12.459220 25721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:12.464920 25733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:12.464931 25730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:12.464934 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:12.465420 25721 server_base.cc:1034] running on GCE node
I20260807 08:38:12.465715 25721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:12.466063 25721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:12.467259 25721 hybrid_clock.cc:648] HybridClock initialized: now 1786091892467229 us; error 57 us; skew 500 ppm
Aug 07 08:38:12 dist-test-slave-h6n0 krb5kdc[25706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:12.470222 25721 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:38:12.470777 25721 webserver.cc:460] Webserver started at http://127.10.207.62:37849/ using document root <none> and password file <none>
I20260807 08:38:12.471025 25721 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:12.471099 25721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:12.471256 25721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:12.472410 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "6c42ef03794e48f78e97cad1a7d977f0"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "2052fb6fe0478c5acdf7ed544a057a98"
server_key_iv: "355c6f6c61decb8e0e715f1c93939167"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.472808 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "6c42ef03794e48f78e97cad1a7d977f0"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "2052fb6fe0478c5acdf7ed544a057a98"
server_key_iv: "355c6f6c61decb8e0e715f1c93939167"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.474277 25721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:38:12.475217 25742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:12.475545 25721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:12.475666 25721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "6c42ef03794e48f78e97cad1a7d977f0"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "2052fb6fe0478c5acdf7ed544a057a98"
server_key_iv: "355c6f6c61decb8e0e715f1c93939167"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.475754 25721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:12.491268 25721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:12.493616 25721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:12.493782 25721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:12.498507 25721 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:37937
I20260807 08:38:12.498665 25805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:37937 every 8 connection(s)
I20260807 08:38:12.499048 25721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:38:12.500305 25807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:12.503300 25807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0: Bootstrap starting.
I20260807 08:38:12.504242 25807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:12.504630 25807 log.cc:824] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:12.505535 25807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0: No bootstrap required, opened a new log
I20260807 08:38:12.507519 25807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42ef03794e48f78e97cad1a7d977f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37937 } }
I20260807 08:38:12.507690 25807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:12.507716 25807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c42ef03794e48f78e97cad1a7d977f0, State: Initialized, Role: FOLLOWER
I20260807 08:38:12.507881 25807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42ef03794e48f78e97cad1a7d977f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37937 } }
I20260807 08:38:12.507977 25807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:38:12.508001 25807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:38:12.508033 25807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:12.508980 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 25721
I20260807 08:38:12.509024 25807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42ef03794e48f78e97cad1a7d977f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37937 } }
I20260807 08:38:12.509120 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:38:12.509151 25807 leader_election.cc:302] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c42ef03794e48f78e97cad1a7d977f0; no voters:
I20260807 08:38:12.509375 25807 leader_election.cc:288] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:38:12.509370 11068 external_mini_cluster.cc:1428] Setting key 0a78d145ca6da670e7ddc77e602f50b2
I20260807 08:38:12.509449 25811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:12.509665 25811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [term 1 LEADER]: Becoming Leader. State: Replica: 6c42ef03794e48f78e97cad1a7d977f0, State: Running, Role: LEADER
I20260807 08:38:12.509716 25807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:38:12.509809 25811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42ef03794e48f78e97cad1a7d977f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37937 } }
I20260807 08:38:12.510308 25811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c42ef03794e48f78e97cad1a7d977f0. Latest consensus state: current_term: 1 leader_uuid: "6c42ef03794e48f78e97cad1a7d977f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42ef03794e48f78e97cad1a7d977f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37937 } } }
I20260807 08:38:12.510424 25811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:12.510329 25812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c42ef03794e48f78e97cad1a7d977f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c42ef03794e48f78e97cad1a7d977f0" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 37937 } } }
I20260807 08:38:12.510704 25812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:38:12.511315 25825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:38:12.512038 25825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:38:12.514324 25825 catalog_manager.cc:1347] Generated new cluster ID: b8ad1decd8bc4530b16b1e1635c27210
I20260807 08:38:12.514389 25825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:38:12 dist-test-slave-h6n0 krb5kdc[25706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:38:12.521391 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:12.510053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.10.207.62:37937)
0807 08:38:12.510690 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:12.510703 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:12.510747 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:38:12.510920 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:12.510931 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:12.511023 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:12.511299 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:12.512137 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.513355 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.514311 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.514900 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.515032 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:12.517418 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:12.517422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:12.517422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:12.517437 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:12.519268 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:12.519647 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:12.519650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:12.519651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:12.519814 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:12.520321 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:12.520323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:12.520324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:12.520623 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:12.520809 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:12.520911 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 08:38:12.521173 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":79,"threads_started":1}
I20260807 08:38:12.528195 25825 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:38:12.528754 25825 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:38:12.532300 25825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0: Generated new TSK 0
I20260807 08:38:12.532539 25825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:38:12.567770 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37937
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:12.664861 25842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:12.665125 25842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:12.665154 25842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:12.666399 25842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:12.666446 25842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:12.666474 25842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:38:12.668100 25842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:12.668386 25842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:12.668686 25842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:12.674571 25850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:12.674811 25842 server_base.cc:1034] running on GCE node
W20260807 08:38:12.674584 25849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:12.674584 25852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:12.675124 25842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:12.675434 25842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:12.676589 25842 hybrid_clock.cc:648] HybridClock initialized: now 1786091892676559 us; error 44 us; skew 500 ppm
Aug 07 08:38:12 dist-test-slave-h6n0 krb5kdc[25706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:12.679773 25842 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:38:12.680313 25842 webserver.cc:460] Webserver started at http://127.10.207.1:34773/ using document root <none> and password file <none>
I20260807 08:38:12.680609 25842 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:12.680684 25842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:12.680850 25842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:12.682098 25842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "9d9b349201434372a28b0772824c7f41"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "5fccc037653bc976d7352affb14dc264"
server_key_iv: "6873b0b832ff584fc5db53f29062f9fa"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.682505 25842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "9d9b349201434372a28b0772824c7f41"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "5fccc037653bc976d7352affb14dc264"
server_key_iv: "6873b0b832ff584fc5db53f29062f9fa"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.684080 25842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:12.685070 25864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:12.685319 25842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:12.685446 25842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "9d9b349201434372a28b0772824c7f41"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "5fccc037653bc976d7352affb14dc264"
server_key_iv: "6873b0b832ff584fc5db53f29062f9fa"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.685528 25842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:12.700546 25842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:12.703347 25842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:12.703536 25842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:12.703845 25842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:12.704248 25842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:12.704308 25842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:12.704345 25842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:12.704387 25842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:12.712852 25842 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:37431
I20260807 08:38:12.712992 26021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:37431 every 8 connection(s)
I20260807 08:38:12.713337 25842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:38:12.715291 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 25842
I20260807 08:38:12.715420 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:38:12.715673 11068 external_mini_cluster.cc:1428] Setting key 75e6ea1d4f11e35cfd1f00d59b67e84e
Aug 07 08:38:12 dist-test-slave-h6n0 krb5kdc[25706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:38:12.725409 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:12.714252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:50997 (local address 127.10.207.62:37937)
0807 08:38:12.714323 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:12.714325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:12.714540 (+ 215us) server_negotiation.cc:407] Connection header received
0807 08:38:12.715332 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:12.715333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:12.715351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:12.715456 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:12.717030 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.717583 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.719051 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.719271 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.719352 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:12.721969 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:12.721971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:12.721972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:12.721980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:12.723818 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:12.724306 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:12.724307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:12.724308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:12.724375 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:12.724651 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:12.724652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:12.724652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:12.724811 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:12.724895 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:12.725197 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 08:38:12.725284 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:38:12.725529 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:12.714185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:37937 (local address 127.10.207.1:50997)
0807 08:38:12.714380 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:12.714429 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:12.715107 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:12.715670 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:12.715676 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:12.716606 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:12.716913 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:12.716920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.717694 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.717695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:12.718929 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:12.718932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.719360 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.719362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:12.719493 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:12.720020 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:12.720031 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:12.721858 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:12.723937 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:12.723945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:12.723948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:12.724215 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:12.724457 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:12.724458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:12.724458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:12.724564 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:12.724892 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:12.724899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:12.725037 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 08:38:12.725142 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 08:38:12.725296 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260807 08:38:12.726313 26022 heartbeater.cc:347] Connected to a master server at 127.10.207.62:37937
I20260807 08:38:12.726454 26022 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
I20260807 08:38:12.726730 26022 heartbeater.cc:511] Master 127.10.207.62:37937 requested a full tablet report, sending...
I20260807 08:38:12.727361 25762 ts_manager.cc:194] Registered new tserver with Master: 9d9b349201434372a28b0772824c7f41 (127.10.207.1:37431)
I20260807 08:38:12.727994 25762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:50997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:38:12.743847 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37937
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:12.835191 26034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:12.835448 26034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:12.835471 26034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:12.836670 26034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:12.836717 26034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:12.836745 26034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:38:12.838289 26034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:12.838586 26034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:12.838908 26034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:12.844667 26044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:12.844651 26045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:12.844651 26047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:12.845098 26034 server_base.cc:1034] running on GCE node
I20260807 08:38:12.845427 26034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:12.845752 26034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:12.846926 26034 hybrid_clock.cc:648] HybridClock initialized: now 1786091892846916 us; error 59 us; skew 500 ppm
Aug 07 08:38:12 dist-test-slave-h6n0 krb5kdc[25706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:12.849785 26034 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:38:12.850297 26034 webserver.cc:460] Webserver started at http://127.10.207.2:41381/ using document root <none> and password file <none>
I20260807 08:38:12.850610 26034 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:12.850685 26034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:12.850840 26034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:12.851972 26034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "908ae87c31434d2cae8fc35172847ff0"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "bb7149b52801b070fa4d07d6a51301ff"
server_key_iv: "fe8e3cf9bae26f91e5ab99416fec018d"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.852377 26034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "908ae87c31434d2cae8fc35172847ff0"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "bb7149b52801b070fa4d07d6a51301ff"
server_key_iv: "fe8e3cf9bae26f91e5ab99416fec018d"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.853824 26034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:38:12.854676 26057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:12.854902 26034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:12.855018 26034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "908ae87c31434d2cae8fc35172847ff0"
format_stamp: "Formatted at 2026-08-07 08:38:12 on dist-test-slave-h6n0"
server_key: "bb7149b52801b070fa4d07d6a51301ff"
server_key_iv: "fe8e3cf9bae26f91e5ab99416fec018d"
server_key_version: "encryptionkey@0"
I20260807 08:38:12.855105 26034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:12.882835 26034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:12.885131 26034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:12.885290 26034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:12.885524 26034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:12.885848 26034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:12.885906 26034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:12.885950 26034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:12.885978 26034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:12.892935 26034 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:41891
I20260807 08:38:12.893003 26230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:41891 every 8 connection(s)
I20260807 08:38:12.893293 26034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:38:12 dist-test-slave-h6n0 krb5kdc[25706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:38:12.900151 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 26034
I20260807 08:38:12.900264 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:38:12.900486 11068 external_mini_cluster.cc:1428] Setting key 915b639f022b9a5ad0672dfc8f392bd5
I20260807 08:38:12.903821 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:12.894105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:37937 (local address 127.10.207.2:50481)
0807 08:38:12.894311 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:12.894349 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:12.894903 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:12.895177 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:12.895181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:12.895744 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:12.895929 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:12.895934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.896678 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.896679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:12.897455 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:12.897457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.897857 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.897858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:12.897981 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:12.898465 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:12.898473 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:12.900253 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:12.902190 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:12.902198 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:12.902200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:12.902525 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:12.902775 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:12.902776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:12.902777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:12.902923 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:12.903200 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:12.903206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:12.903367 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 08:38:12.903492 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 08:38:12.903660 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":110,"threads_started":1}
I20260807 08:38:12.903887 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:12.894147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:50481 (local address 127.10.207.62:37937)
0807 08:38:12.894221 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:12.894223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:12.894443 (+ 220us) server_negotiation.cc:407] Connection header received
0807 08:38:12.895042 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:12.895043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:12.895058 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:12.895110 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:12.896034 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.896571 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.897557 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:12.897769 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:12.897849 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:12.900406 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:12.900411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:12.900411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:12.900426 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:12.902091 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:12.902634 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:12.902635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:12.902635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:12.902694 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:12.903006 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:12.903007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:12.903007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:12.903131 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:12.903183 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:12.903613 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 08:38:12.903736 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:38:12.904595 26231 heartbeater.cc:347] Connected to a master server at 127.10.207.62:37937
I20260807 08:38:12.904695 26231 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:12.904882 26231 heartbeater.cc:511] Master 127.10.207.62:37937 requested a full tablet report, sending...
I20260807 08:38:12.905303 25762 ts_manager.cc:194] Registered new tserver with Master: 908ae87c31434d2cae8fc35172847ff0 (127.10.207.2:41891)
I20260807 08:38:12.905849 25762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:50481
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:38:12.929463 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37937
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:38:13.025632 26242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:38:13.025910 26242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:38:13.025946 26242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:38:13.027171 26242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:38:13.027237 26242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:38:13.027284 26242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:38:13.028846 26242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:38:13.029157 26242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:38:13.029418 26242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:38:13.035295 26249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:13.035310 26250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:38:13.035310 26252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:38:13.035490 26242 server_base.cc:1034] running on GCE node
I20260807 08:38:13.035918 26242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:38:13.036249 26242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:38:13.037421 26242 hybrid_clock.cc:648] HybridClock initialized: now 1786091893037416 us; error 47 us; skew 500 ppm
Aug 07 08:38:13 dist-test-slave-h6n0 krb5kdc[25706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091893, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:38:13.040419 26242 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:38:13.040992 26242 webserver.cc:460] Webserver started at http://127.10.207.3:37621/ using document root <none> and password file <none>
I20260807 08:38:13.041254 26242 fs_manager.cc:359] Metadata directory not provided
I20260807 08:38:13.041328 26242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:38:13.041489 26242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:38:13.042763 26242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "d36e76ce2e524318aa74e17bccb0a8f3"
format_stamp: "Formatted at 2026-08-07 08:38:13 on dist-test-slave-h6n0"
server_key: "b74307cbe6cc97b0986131ef0ab47bc4"
server_key_iv: "da3a50ea58ef9556717079ab0fa1c825"
server_key_version: "encryptionkey@0"
I20260807 08:38:13.043170 26242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "d36e76ce2e524318aa74e17bccb0a8f3"
format_stamp: "Formatted at 2026-08-07 08:38:13 on dist-test-slave-h6n0"
server_key: "b74307cbe6cc97b0986131ef0ab47bc4"
server_key_iv: "da3a50ea58ef9556717079ab0fa1c825"
server_key_version: "encryptionkey@0"
I20260807 08:38:13.044860 26242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:38:13.045745 26264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:13.045974 26242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:38:13.046108 26242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "d36e76ce2e524318aa74e17bccb0a8f3"
format_stamp: "Formatted at 2026-08-07 08:38:13 on dist-test-slave-h6n0"
server_key: "b74307cbe6cc97b0986131ef0ab47bc4"
server_key_iv: "da3a50ea58ef9556717079ab0fa1c825"
server_key_version: "encryptionkey@0"
I20260807 08:38:13.046195 26242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:38:13.087864 26242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:38:13.090426 26242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:38:13.090633 26242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:38:13.090939 26242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:38:13.091339 26242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:38:13.091399 26242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:13.091449 26242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:38:13.091473 26242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:38:13.098958 26242 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34491
I20260807 08:38:13.099107 26430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34491 every 8 connection(s)
I20260807 08:38:13.099462 26242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:38:13.106868 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 26242
I20260807 08:38:13.106957 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:38:13.107187 11068 external_mini_cluster.cc:1428] Setting key 9d692de1cce6bd9ab24b1bc5209e51ee
Aug 07 08:38:13 dist-test-slave-h6n0 krb5kdc[25706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091893, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:38:13.111486 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.100624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:53795 (local address 127.10.207.62:37937)
0807 08:38:13.100724 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.100727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.101012 (+ 285us) server_negotiation.cc:407] Connection header received
0807 08:38:13.101735 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.101737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.101750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.101823 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:13.103027 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.103548 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.104708 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.104928 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.105016 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.107967 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:13.107968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:13.107969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:13.107979 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:13.109743 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:13.110223 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:13.110224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:13.110224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:13.110283 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:13.110764 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:13.110764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:13.110765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:13.110950 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:13.111010 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.111329 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.111412 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:38:13.111696 26435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.100588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:37937 (local address 127.10.207.3:53795)
0807 08:38:13.100841 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:13.100887 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:13.101544 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:13.101896 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:13.101900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:13.102652 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:38:13.102902 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.102908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.103657 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.103657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.104574 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.104577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.105030 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.105030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.105172 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.105947 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:38:13.105956 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:38:13.107866 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:38:13.109841 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:13.109849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:13.109852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:13.110139 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:13.110400 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:38:13.110402 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:38:13.110403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:38:13.110683 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:38:13.111023 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:38:13.111029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:38:13.111182 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 08:38:13.111279 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:38:13.111463 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":156,"threads_started":1}
I20260807 08:38:13.112237 26431 heartbeater.cc:347] Connected to a master server at 127.10.207.62:37937
I20260807 08:38:13.112341 26431 heartbeater.cc:464] Registering TS with master...
I20260807 08:38:13.112514 26431 heartbeater.cc:511] Master 127.10.207.62:37937 requested a full tablet report, sending...
I20260807 08:38:13.112944 25762 ts_manager.cc:194] Registered new tserver with Master: d36e76ce2e524318aa74e17bccb0a8f3 (127.10.207.3:34491)
I20260807 08:38:13.113380 25762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:53795
I20260807 08:38:13.120019 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:38:13.128468 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.121772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.10.207.62:37937)
0807 08:38:13.121861 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.121863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.121991 (+ 128us) server_negotiation.cc:407] Connection header received
0807 08:38:13.122020 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.122021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.122032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.122080 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:38:13.122782 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.123317 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.124081 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.124280 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.124343 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.125647 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:38:13.125648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:38:13.125648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:38:13.125655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:38:13.127326 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:13.127687 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:13.127688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:13.127688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:13.127747 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:38:13.127941 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:38:13.127942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:38:13.127942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:38:13.128124 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:38:13.128212 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.128321 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.128392 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:38:13.130776 25762 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:38:13.132645 25762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:38:13.140164 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.135920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41891 (local address 127.0.0.1:41564)
0807 08:38:13.136241 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:13.136253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:13.136292 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:13.136593 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:13.136594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:13.136596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:13.136657 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.136659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.137598 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.137600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.139023 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.139027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.139786 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.139787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.139869 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.139884 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 08:38:13.139991 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 08:38:13.140064 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":5,"thread_start_us":103,"threads_started":1}
I20260807 08:38:13.140321 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.135818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40332 (local address 127.10.207.3:34491)
0807 08:38:13.136554 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.136558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.136569 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:38:13.136609 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.136612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.136640 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.136759 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:13.136880 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.137931 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.139122 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.139896 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.140006 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.140073 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.140183 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.140225 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":133,"threads_started":1}
I20260807 08:38:13.140311 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.135696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34491 (local address 127.0.0.1:40332)
0807 08:38:13.136071 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:13.136084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:13.136139 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:13.136769 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:13.136770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:13.136772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:13.136809 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.136810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.138027 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.138028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.139019 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.139022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.140021 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.140022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.140086 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.140089 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:38:13.140148 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:38:13.140198 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":124,"threads_started":1}
I20260807 08:38:13.140720 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.135683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:37431 (local address 127.0.0.1:52768)
0807 08:38:13.136022 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:13.136065 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:13.136139 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:13.136423 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:13.136428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:13.136431 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:13.136520 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.136523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.137953 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.137954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.139154 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.139156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.140037 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.140038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.140113 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.140149 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:38:13.140187 (+ 38us) client_negotiation.cc:240] Negotiation successful
0807 08:38:13.140244 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":78,"threads_started":1}
I20260807 08:38:13.140816 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.136083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.10.207.2:41891)
0807 08:38:13.136343 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.136348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.136359 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:38:13.136398 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.136400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.136415 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.136493 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:13.136741 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.137489 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.139154 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.139690 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.139783 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.139844 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.140694 (+ 850us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.140741 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":143,"threads_started":1}
I20260807 08:38:13.141191 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.135804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.10.207.1:37431)
0807 08:38:13.136167 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.136174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.136190 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:38:13.136251 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.136253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.136275 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.136381 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:13.136624 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.137824 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.139228 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.139932 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.140923 (+ 991us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.140991 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.141018 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.141071 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":102,"threads_started":1}
I20260807 08:38:13.141214 26332 tablet_service.cc:1467] Processing CreateTablet for tablet 567b056146b040b5ad7561aa271c79a0 (DEFAULT_TABLE table=test-table [id=33c1bd4c30fb4166aa0b2fe2fc20a3be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:13.141372 26138 tablet_service.cc:1467] Processing CreateTablet for tablet 567b056146b040b5ad7561aa271c79a0 (DEFAULT_TABLE table=test-table [id=33c1bd4c30fb4166aa0b2fe2fc20a3be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:13.141618 26332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567b056146b040b5ad7561aa271c79a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:13.141675 26138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567b056146b040b5ad7561aa271c79a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:13.143231 25934 tablet_service.cc:1467] Processing CreateTablet for tablet 567b056146b040b5ad7561aa271c79a0 (DEFAULT_TABLE table=test-table [id=33c1bd4c30fb4166aa0b2fe2fc20a3be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:38:13.143692 25934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567b056146b040b5ad7561aa271c79a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:38:13.145159 26457 tablet_bootstrap.cc:492] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Bootstrap starting.
I20260807 08:38:13.145437 26456 tablet_bootstrap.cc:492] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Bootstrap starting.
I20260807 08:38:13.146293 26456 tablet_bootstrap.cc:654] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:13.146977 26456 log.cc:824] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:13.147202 26457 tablet_bootstrap.cc:654] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:13.147569 26460 tablet_bootstrap.cc:492] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: Bootstrap starting.
I20260807 08:38:13.147850 26457 log.cc:824] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:13.148303 26460 tablet_bootstrap.cc:654] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: Neither blocks nor log segments found. Creating new log.
I20260807 08:38:13.148869 26460 log.cc:824] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: Log is configured to *not* fsync() on all Append() calls
I20260807 08:38:13.148910 26457 tablet_bootstrap.cc:492] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: No bootstrap required, opened a new log
I20260807 08:38:13.148972 26457 ts_tablet_manager.cc:1397] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:38:13.149768 26456 tablet_bootstrap.cc:492] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: No bootstrap required, opened a new log
I20260807 08:38:13.149868 26456 ts_tablet_manager.cc:1397] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260807 08:38:13.149906 26460 tablet_bootstrap.cc:492] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: No bootstrap required, opened a new log
I20260807 08:38:13.149987 26460 ts_tablet_manager.cc:1397] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:38:13.151082 26457 raft_consensus.cc:348] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.151254 26457 raft_consensus.cc:374] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:13.151289 26457 raft_consensus.cc:729] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36e76ce2e524318aa74e17bccb0a8f3, State: Initialized, Role: FOLLOWER
I20260807 08:38:13.151461 26457 consensus_queue.cc:260] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.151752 26457 ts_tablet_manager.cc:1428] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:13.151883 26431 heartbeater.cc:503] Master 127.10.207.62:37937 was elected leader, sending a full tablet report...
I20260807 08:38:13.152241 26460 raft_consensus.cc:348] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.152376 26460 raft_consensus.cc:374] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:13.152400 26460 raft_consensus.cc:729] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d9b349201434372a28b0772824c7f41, State: Initialized, Role: FOLLOWER
I20260807 08:38:13.152295 26456 raft_consensus.cc:348] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.152495 26456 raft_consensus.cc:374] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:38:13.152534 26456 raft_consensus.cc:729] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908ae87c31434d2cae8fc35172847ff0, State: Initialized, Role: FOLLOWER
I20260807 08:38:13.152508 26460 consensus_queue.cc:260] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.152678 26456 consensus_queue.cc:260] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.152779 26460 ts_tablet_manager.cc:1428] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:38:13.152805 26022 heartbeater.cc:503] Master 127.10.207.62:37937 was elected leader, sending a full tablet report...
I20260807 08:38:13.153018 26456 ts_tablet_manager.cc:1428] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260807 08:38:13.153023 26231 heartbeater.cc:503] Master 127.10.207.62:37937 was elected leader, sending a full tablet report...
I20260807 08:38:13.205271 26466 raft_consensus.cc:482] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:38:13.205404 26466 raft_consensus.cc:504] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.206157 26466 leader_election.cc:288] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36e76ce2e524318aa74e17bccb0a8f3 (127.10.207.3:34491), 9d9b349201434372a28b0772824c7f41 (127.10.207.1:37431)
I20260807 08:38:13.209407 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.206192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:59205 (local address 127.10.207.1:37431)
0807 08:38:13.206244 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.206246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.206259 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:38:13.206376 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.206378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.206390 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.206459 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:13.206716 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.207289 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.208498 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.208962 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.209022 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.209030 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.209286 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.209329 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:38:13.209398 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.206111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:37431 (local address 127.10.207.2:59205)
0807 08:38:13.206193 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:13.206210 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:13.206273 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:13.206487 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:13.206488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:13.206491 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:13.206619 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.206622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.207395 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.207396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.208391 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.208394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.209066 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.209067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.209163 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.209169 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:38:13.209248 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 08:38:13.209312 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 08:38:13.209615 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.206185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34491 (local address 127.10.207.2:47993)
0807 08:38:13.206434 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:13.206448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:38:13.206552 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:38:13.206718 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:38:13.206719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:38:13.206721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:38:13.206804 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.206807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.207710 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.207711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.208595 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:38:13.208597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.209301 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.209302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:38:13.209388 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.209394 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:38:13.209463 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:38:13.209517 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":179,"threads_started":1}
I20260807 08:38:13.209653 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.206405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:47993 (local address 127.10.207.3:34491)
0807 08:38:13.206463 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.206465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.206511 (+ 46us) server_negotiation.cc:407] Connection header received
0807 08:38:13.206617 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.206618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.206629 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.206691 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:38:13.206925 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.207583 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.208707 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.209208 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.209281 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.209291 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.209500 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.209580 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:38:13.209918 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567b056146b040b5ad7561aa271c79a0" candidate_uuid: "908ae87c31434d2cae8fc35172847ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d9b349201434372a28b0772824c7f41" is_pre_election: true
I20260807 08:38:13.210119 26357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567b056146b040b5ad7561aa271c79a0" candidate_uuid: "908ae87c31434d2cae8fc35172847ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" is_pre_election: true
I20260807 08:38:13.210220 25965 raft_consensus.cc:2454] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 908ae87c31434d2cae8fc35172847ff0 in term 0.
I20260807 08:38:13.210335 26357 raft_consensus.cc:2454] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 908ae87c31434d2cae8fc35172847ff0 in term 0.
I20260807 08:38:13.210553 26058 leader_election.cc:302] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908ae87c31434d2cae8fc35172847ff0, 9d9b349201434372a28b0772824c7f41; no voters:
I20260807 08:38:13.210722 26466 raft_consensus.cc:2781] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:38:13.210784 26466 raft_consensus.cc:482] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:38:13.210829 26466 raft_consensus.cc:3037] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:13.211740 26466 raft_consensus.cc:504] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
I20260807 08:38:13.211915 26466 leader_election.cc:288] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [CANDIDATE]: Term 1 election: Requested vote from peers d36e76ce2e524318aa74e17bccb0a8f3 (127.10.207.3:34491), 9d9b349201434372a28b0772824c7f41 (127.10.207.1:37431)
I20260807 08:38:13.212102 26357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567b056146b040b5ad7561aa271c79a0" candidate_uuid: "908ae87c31434d2cae8fc35172847ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36e76ce2e524318aa74e17bccb0a8f3"
I20260807 08:38:13.212133 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567b056146b040b5ad7561aa271c79a0" candidate_uuid: "908ae87c31434d2cae8fc35172847ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d9b349201434372a28b0772824c7f41"
I20260807 08:38:13.212200 26357 raft_consensus.cc:3037] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:13.212222 25965 raft_consensus.cc:3037] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:38:13.213013 26357 raft_consensus.cc:2454] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 908ae87c31434d2cae8fc35172847ff0 in term 1.
I20260807 08:38:13.213240 26060 leader_election.cc:302] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908ae87c31434d2cae8fc35172847ff0, d36e76ce2e524318aa74e17bccb0a8f3; no voters:
I20260807 08:38:13.213335 26466 raft_consensus.cc:2781] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:38:13.213366 25965 raft_consensus.cc:2454] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 908ae87c31434d2cae8fc35172847ff0 in term 1.
I20260807 08:38:13.213518 26466 raft_consensus.cc:686] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 1 LEADER]: Becoming Leader. State: Replica: 908ae87c31434d2cae8fc35172847ff0, State: Running, Role: LEADER
I20260807 08:38:13.213621 26466 consensus_queue.cc:237] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } }
W20260807 08:38:13.214149 26024 tablet.cc:2369] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:38:13.214697 25760 catalog_manager.cc:5495] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 908ae87c31434d2cae8fc35172847ff0 (127.10.207.2). New cstate: current_term: 1 leader_uuid: "908ae87c31434d2cae8fc35172847ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "908ae87c31434d2cae8fc35172847ff0" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41891 } health_report { overall_health: HEALTHY } } }
I20260807 08:38:13.237432 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:38:13.234849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41580 (local address 127.10.207.2:41891)
0807 08:38:13.234931 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 08:38:13.234933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:38:13.234955 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:38:13.235002 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:38:13.235003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:38:13.235023 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:38:13.235089 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:38:13.235243 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.235760 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.236586 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:38:13.236788 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:38:13.236847 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:38:13.237040 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:38:13.237153 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:38:13.237223 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:38:13.237317 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 08:38:13.237338 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:38:13.239709 26357 raft_consensus.cc:1261] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 1 FOLLOWER]: Refusing update from remote peer 908ae87c31434d2cae8fc35172847ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:13.239857 25965 raft_consensus.cc:1261] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 1 FOLLOWER]: Refusing update from remote peer 908ae87c31434d2cae8fc35172847ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:38:13.240151 26468 consensus_queue.cc:1035] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36e76ce2e524318aa74e17bccb0a8f3" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:13.240324 26466 consensus_queue.cc:1035] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d9b349201434372a28b0772824c7f41" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:38:13.243420 26472 mvcc.cc:205] Tried to move back new op lower bound from 7315832394707025920 to 7315832394603823104. Current Snapshot: MvccSnapshot[applied={T|T < 7315832394707025920}]
I20260807 08:38:13.246629 25760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:13.246773 25760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:38:13.247768 25760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c42ef03794e48f78e97cad1a7d977f0: Sending DeleteTablet for 3 replicas of tablet 567b056146b040b5ad7561aa271c79a0
I20260807 08:38:13.248196 26138 tablet_service.cc:1514] Processing DeleteTablet for tablet 567b056146b040b5ad7561aa271c79a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:13 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:41564
I20260807 08:38:13.248221 25934 tablet_service.cc:1514] Processing DeleteTablet for tablet 567b056146b040b5ad7561aa271c79a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:13 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:52768
I20260807 08:38:13.248257 26332 tablet_service.cc:1514] Processing DeleteTablet for tablet 567b056146b040b5ad7561aa271c79a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:38:13 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:40332
I20260807 08:38:13.248473 26489 tablet_replica.cc:330] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: stopping tablet replica
I20260807 08:38:13.248605 26489 raft_consensus.cc:2229] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:38:13.248692 26488 tablet_replica.cc:330] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41: stopping tablet replica
I20260807 08:38:13.248768 26489 raft_consensus.cc:2258] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:13.248783 26488 raft_consensus.cc:2229] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:13.248847 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 25842
I20260807 08:38:13.248880 26488 raft_consensus.cc:2258] T 567b056146b040b5ad7561aa271c79a0 P 9d9b349201434372a28b0772824c7f41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:13.251986 26490 tablet_replica.cc:330] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: stopping tablet replica
I20260807 08:38:13.252105 26490 raft_consensus.cc:2229] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:38:13.252213 26490 raft_consensus.cc:2258] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:38:13.254588 26490 ts_tablet_manager.cc:1905] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:13.256819 26490 ts_tablet_manager.cc:1918] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:13.256893 26490 log.cc:1196] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/567b056146b040b5ad7561aa271c79a0
I20260807 08:38:13.257231 26490 ts_tablet_manager.cc:1939] T 567b056146b040b5ad7561aa271c79a0 P d36e76ce2e524318aa74e17bccb0a8f3: Deleting consensus metadata
I20260807 08:38:13.257795 25746 catalog_manager.cc:4841] TS d36e76ce2e524318aa74e17bccb0a8f3 (127.10.207.3:34491): tablet 567b056146b040b5ad7561aa271c79a0 (table test-table [id=33c1bd4c30fb4166aa0b2fe2fc20a3be]) successfully deleted
I20260807 08:38:13.258927 26489 ts_tablet_manager.cc:1905] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:38:13.260763 26489 ts_tablet_manager.cc:1918] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:38:13.260905 26489 log.cc:1196] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/567b056146b040b5ad7561aa271c79a0
W20260807 08:38:13.260962 25743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:37431 (error 108)
I20260807 08:38:13.261274 26489 ts_tablet_manager.cc:1939] T 567b056146b040b5ad7561aa271c79a0 P 908ae87c31434d2cae8fc35172847ff0: Deleting consensus metadata
W20260807 08:38:13.261778 26450 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:38:13.261485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:37431 (local address 127.0.0.1:52782)
0807 08:38:13.261673 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:38:13.261715 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:37431: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":142}
W20260807 08:38:13.261986 25743 catalog_manager.cc:4584] TS 9d9b349201434372a28b0772824c7f41 (127.10.207.1:37431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 567b056146b040b5ad7561aa271c79a0: Network error: Client connection negotiation failed: client connection to 127.10.207.1:37431: connect: Connection reset by peer (error 104)
I20260807 08:38:13.262032 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 26034
I20260807 08:38:13.262111 25743 catalog_manager.cc:4841] TS 908ae87c31434d2cae8fc35172847ff0 (127.10.207.2:41891): tablet 567b056146b040b5ad7561aa271c79a0 (table test-table [id=33c1bd4c30fb4166aa0b2fe2fc20a3be]) successfully deleted
I20260807 08:38:13.277254 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 26242
I20260807 08:38:13.290036 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 25721
2026-08-07T08:38:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2386 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:38:13 dist-test-slave-h6n0 krb5kdc[26495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:38:13 dist-test-slave-h6n0 krb5kdc[26495](info): set up 2 sockets
Aug 07 08:38:13 dist-test-slave-h6n0 krb5kdc[26495](info): commencing operation
krb5kdc: starting...
W20260807 08:38:14.560500 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:38:14 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:38:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/rangeradmin_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/rangeradmin_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/rangerlookup_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/rangerlookup_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
I20260807 08:38:14.749589 11068 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDqD6RB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/postgres -l logfile start
2026-08-07 08:38:17.215 UTC [26536] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:38:17.215 UTC [26536] LOG: listening on IPv4 address "127.10.207.20", port 34983
2026-08-07 08:38:17.218 UTC [26536] LOG: listening on Unix socket "/tmp/.s.PGSQL.34983"
2026-08-07 08:38:17.239 UTC [26541] LOG: database system was shut down at 2026-08-07 08:38:15 UTC
2026-08-07 08:38:17.242 UTC [26536] LOG: database system is ready to accept connections
I20260807 08:38:18.449316 11068 mini_postgres.cc:96] Postgres bound to 34983
2026-08-07 08:38:18.457 UTC [26552] FATAL: database "slave" does not exist
127.10.207.20:34983 - accepting connections
I20260807 08:38:18.458680 11068 mini_ranger.cc:161] Starting Ranger...
I20260807 08:38:18.475023 11068 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:38:18.950050 11068 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:38:18.950140 11068 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-admin
2026-08-07 08:38:19,299 [I] DB FLAVOR :POSTGRES
2026-08-07 08:38:19,300 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:38:19,300 [I] Checking connection..
2026-08-07 08:38:19,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:38:19,575 [I] Checking connection passed.
2026-08-07 08:38:19,575 [I] --------- Verifying version history table ---------
2026-08-07 08:38:19,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:38:19,854 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:38:19,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:38:20,119 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:38:20,119 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:38:20,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:38:20.368 UTC [26648] WARNING: there is no transaction in progress
2026-08-07 08:38:20,379 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:38:20,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:38:20,658 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:38:20,658 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:38:20,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:38:20,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:38:21,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-07 08:38:21,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:38:21,750 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:38:21,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:38:22,036 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:38:22,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:38:22,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:38:22,593 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:38:22,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:38:23.595 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.612 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.628 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.642 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.661 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.682 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.749 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.756 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.771 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.782 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.793 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.807 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.816 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.822 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.828 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:23.834 UTC [26812] WARNING: there is no transaction in progress
2026-08-07 08:38:24,466 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:38:24,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:38:24,735 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:38:24,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:38:24,998 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:38:24,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:38:25,291 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:38:25,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:38:25,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:38:25,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-07 08:38:26,112 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:38:26,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:38:26,382 [I] DB_PATCHES have already been applied
I20260807 08:38:26.389119 11068 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDqD6RB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:38:26.389194 11068 mini_ranger.cc:193] Using host: 127.10.207.20
I20260807 08:38:26.393813 11068 mini_ranger.cc:227] Ranger admin URL: http://127.10.207.20:33189
Aug 07, 2026 8:38:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:38:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:38:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:38:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:38:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:38:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:38:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:38:27.785459 11068 mini_ranger.cc:160] Time spent starting Ranger: real 9.327s user 0.000s sys 0.013s
Aug 07, 2026 8:38:48 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:38:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 8:38:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:38:48 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:38:49 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:38:50.407918 11068 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/rangerkms_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/rangerkms_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.207.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.207.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:38:50 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091930, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:38:50.531334 11068 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:38:50.544796 11068 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:38:51.369443 11068 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:38:51.369527 11068 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms
I20260807 08:38:51.369550 11068 mini_ranger_kms.cc:227] Using postgres at 127.10.207.20:34983
2026-08-07 08:38:52,036 [I] DB FLAVOR :POSTGRES
2026-08-07 08:38:52,036 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:38:52,036 [I] Checking connection
2026-08-07 08:38:52,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:38:52,381 [I] connection success
2026-08-07 08:38:52,382 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:38:52,382 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:38:52,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:38:52,642 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:38:52,642 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:38:52,642 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:38:52,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:34983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:38:52,926 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:38:53.457937 11068 mini_ranger_kms.cc:310] Created kms service
I20260807 08:38:53.649207 11068 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:38:53.718243 11068 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:38:53 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091930, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:38:54.180785 11068 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:38:54.180935 11068 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:38:54.180960 11068 mini_ranger_kms.cc:240] Using host: 127.10.207.20
I20260807 08:38:54.185447 11068 mini_ranger_kms.cc:276] Ranger KMS PID: 27088
I20260807 08:38:54.185649 11068 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.207.20:45829
Aug 07, 2026 8:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:38:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:38:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 08:38:57.091187 11068 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.560s user 0.000s sys 0.635s
I20260807 08:38:57.091317 11068 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:38:57.091398 11068 mini_ranger_kms.cc:401] 127.10.207.20:45829/kms/v1/keys
Aug 07 08:38:57 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091937, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.207.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 08:38:57 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091937, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.207.20@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
Aug 07 08:38:58 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091937, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.207.20@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
Aug 07 08:39:00 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:39:00.831704 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43615
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43615
--ranger_config_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:45829/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:00.970501 27168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:00.970777 27168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:00.970801 27168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:00.972194 27168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:00.972242 27168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:00.972257 27168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:00.972270 27168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:00.972282 27168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:00.973999 27168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.207.20:45829/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43615
--ranger_config_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:00.974232 27168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:00.974498 27168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:00.981851 27174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:00.981899 27176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:00.981910 27168 server_base.cc:1034] running on GCE node
W20260807 08:39:00.981930 27179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:00.982492 27168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:00.982863 27168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:00.984050 27168 hybrid_clock.cc:648] HybridClock initialized: now 1786091940984019 us; error 51 us; skew 500 ppm
Aug 07 08:39:00 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091940, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:00.991693 27168 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:39:00.992358 27168 webserver.cc:460] Webserver started at http://127.10.207.62:46247/ using document root <none> and password file <none>
I20260807 08:39:00.992624 27168 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:00.992682 27168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:00.992818 27168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:00 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091940, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:01.058291 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "8b1ced7ff51b4e8aa539f984be1b852b"
format_stamp: "Formatted at 2026-08-07 08:39:01 on dist-test-slave-h6n0"
server_key: "68d22b9bf7ec09e51289a1111478f36b"
server_key_iv: "034c3f3ef84f7eef532005af81c9fcea"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:01.058974 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "8b1ced7ff51b4e8aa539f984be1b852b"
format_stamp: "Formatted at 2026-08-07 08:39:01 on dist-test-slave-h6n0"
server_key: "68d22b9bf7ec09e51289a1111478f36b"
server_key_iv: "034c3f3ef84f7eef532005af81c9fcea"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:01.060720 27168 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:39:01.078385 27189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:01.078754 27168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:39:01.078946 27168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "8b1ced7ff51b4e8aa539f984be1b852b"
format_stamp: "Formatted at 2026-08-07 08:39:01 on dist-test-slave-h6n0"
server_key: "68d22b9bf7ec09e51289a1111478f36b"
server_key_iv: "034c3f3ef84f7eef532005af81c9fcea"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:01.079082 27168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:01.089799 27168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:01.092247 27168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:01.092453 27168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:01.101131 27168 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43615
I20260807 08:39:01.101219 27280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43615 every 8 connection(s)
I20260807 08:39:01.101660 27168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:01.101971 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 27168
I20260807 08:39:01.102097 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:39:01.102324 11068 external_mini_cluster.cc:1428] Setting key 42f801b1ddc623cf38a38b3b3e52d941
I20260807 08:39:01.102772 27281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:01.106959 27281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b: Bootstrap starting.
I20260807 08:39:01.107991 27281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:01.108497 27281 log.cc:824] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:01.109488 27281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b: No bootstrap required, opened a new log
Aug 07 08:39:01 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:01.111974 27281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43615 } }
I20260807 08:39:01.112136 27281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:01.112202 27281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1ced7ff51b4e8aa539f984be1b852b, State: Initialized, Role: FOLLOWER
I20260807 08:39:01.112362 27281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43615 } }
I20260807 08:39:01.112479 27281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:01.112527 27281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:01.112581 27281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:01.113343 27281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43615 } }
I20260807 08:39:01.113490 27281 leader_election.cc:302] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b1ced7ff51b4e8aa539f984be1b852b; no voters:
I20260807 08:39:01.113703 27281 leader_election.cc:288] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:01.113763 27288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:01.114239 27288 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [term 1 LEADER]: Becoming Leader. State: Replica: 8b1ced7ff51b4e8aa539f984be1b852b, State: Running, Role: LEADER
I20260807 08:39:01.114428 27288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43615 } }
I20260807 08:39:01.114511 27281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:01.114926 27288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43615 } } }
I20260807 08:39:01.115093 27288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:01.115337 27288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b1ced7ff51b4e8aa539f984be1b852b. Latest consensus state: current_term: 1 leader_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1ced7ff51b4e8aa539f984be1b852b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43615 } } }
I20260807 08:39:01.115465 27288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:01.115691 27293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:01.117184 27293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:01.117285 27284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:01.103085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.10.207.62:43615)
0807 08:39:01.103350 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:01.103362 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:01.103404 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:39:01.104386 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:01.104398 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:01.104471 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:01.104645 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:01.105537 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:01.107077 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:01.108249 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:01.108917 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:01.109054 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:01.112390 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:01.112397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:01.112397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:01.112414 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:01.115143 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:01.115631 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:01.115634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:01.115634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:01.115740 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:01.115979 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:01.115980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:01.115980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:01.116289 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:01.116826 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:01.116898 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:39:01.117031 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":196,"thread_start_us":127,"threads_started":1}
I20260807 08:39:01.118062 27281 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:39:01.120217 27293 catalog_manager.cc:1347] Generated new cluster ID: 8404e010e84e402eae0401ec2def34ca
I20260807 08:39:01.120276 27293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:01.131392 27293 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:01.132357 27293 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:01.139665 27293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b: Generated new TSK 0
I20260807 08:39:01.139973 27293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:39:02.172994 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:45829/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43615
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:02.347016 27349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:02.347304 27349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:02.347337 27349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:02.348783 27349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:02.348830 27349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:02.348861 27349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:39:02.351084 27349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.207.20:45829/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:02.351488 27349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:02.351811 27349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:02.359683 27357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:02.360868 27362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.361173 27349 server_base.cc:1034] running on GCE node
W20260807 08:39:02.361367 27358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.361963 27349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:02.362282 27349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:02.363453 27349 hybrid_clock.cc:648] HybridClock initialized: now 1786091942363410 us; error 57 us; skew 500 ppm
Aug 07 08:39:02 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:02.368934 27349 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:39:02.370044 27349 webserver.cc:460] Webserver started at http://127.10.207.1:43389/ using document root <none> and password file <none>
I20260807 08:39:02.370436 27349 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:02.370571 27349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:02.370767 27349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:02 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:02.390095 27349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "649b7ad34be24d24bbd991901c4a0169"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h6n0"
server_key: "eb3b82ea872a09fe452f51bda6e43281"
server_key_iv: "d5ccb6eea142b4ee62958d22cec247ad"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:02.391765 27349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "649b7ad34be24d24bbd991901c4a0169"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h6n0"
server_key: "eb3b82ea872a09fe452f51bda6e43281"
server_key_iv: "d5ccb6eea142b4ee62958d22cec247ad"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:02.396116 27349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.001s
I20260807 08:39:02.411193 27374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.411468 27349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:39:02.411588 27349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "649b7ad34be24d24bbd991901c4a0169"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h6n0"
server_key: "eb3b82ea872a09fe452f51bda6e43281"
server_key_iv: "d5ccb6eea142b4ee62958d22cec247ad"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:02.411720 27349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:02.437997 27349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:02.440868 27349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:02.441118 27349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:02.441597 27349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:02.442214 27349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:02.442355 27349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.442414 27349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:02.442493 27349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.460322 27349 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:40887
I20260807 08:39:02.460631 27565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:40887 every 8 connection(s)
I20260807 08:39:02.460886 27349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:39:02.463434 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 27349
I20260807 08:39:02.463587 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:39:02.463829 11068 external_mini_cluster.cc:1428] Setting key c111a8c0ad0023d46f057b978cce18ab
Aug 07 08:39:02 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
I20260807 08:39:02.481724 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.461987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43615 (local address 127.10.207.1:45867)
0807 08:39:02.462268 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:02.462326 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:02.463175 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:02.463610 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:02.463616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:02.463971 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:02.464355 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.464368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.465443 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.465445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.467040 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.467045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.467675 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.467676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.467855 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.468627 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:02.468637 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:02.470740 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:02.479539 (+ 8799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.479546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.479549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.479906 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.480216 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.480217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.480217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.480363 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.480788 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:02.480802 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:02.481038 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 08:39:02.481175 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 08:39:02.481391 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":122,"threads_started":1}
I20260807 08:39:02.482882 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.462060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:45867 (local address 127.10.207.62:43615)
0807 08:39:02.462295 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:02.462298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:02.462469 (+ 171us) server_negotiation.cc:407] Connection header received
0807 08:39:02.463348 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:02.463350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:02.463368 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:02.463477 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:02.464470 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.465307 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.467167 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.467511 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.467591 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.470881 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:02.470883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:02.470884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:02.470894 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:02.479385 (+ 8491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.480037 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.480039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.480039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.480126 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.480470 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.480471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.480472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.480665 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:02.480743 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:02.482635 (+ 1892us) server_negotiation.cc:299] Negotiation successful
0807 08:39:02.482746 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":154,"threads_started":1}
I20260807 08:39:02.483613 27567 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43615
I20260807 08:39:02.483727 27567 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:02.483955 27567 heartbeater.cc:511] Master 127.10.207.62:43615 requested a full tablet report, sending...
I20260807 08:39:02.484797 27217 ts_manager.cc:194] Registered new tserver with Master: 649b7ad34be24d24bbd991901c4a0169 (127.10.207.1:40887)
I20260807 08:39:02.485765 27217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:45867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:39:02.507385 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:45829/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43615
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:39:02 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 08:39:02.679239 27574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:02.679699 27574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:02.679803 27574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:02.681973 27574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:02.682089 27574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:02.682183 27574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:39:02.684921 27574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.207.20:45829/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:02.685480 27574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:02.685880 27574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:02.695375 27594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:02.695598 27585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:02.696276 27588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:02.696491 27574 server_base.cc:1034] running on GCE node
I20260807 08:39:02.696885 27574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:02.697265 27574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:02.698459 27574 hybrid_clock.cc:648] HybridClock initialized: now 1786091942698420 us; error 56 us; skew 500 ppm
Aug 07 08:39:02 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:02.703516 27574 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:39:02.704329 27574 webserver.cc:460] Webserver started at http://127.10.207.2:35671/ using document root <none> and password file <none>
I20260807 08:39:02.704764 27574 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:02.704855 27574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:02.705076 27574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:02 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:02.721807 27574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "296db31b064f47d39f52926ebef10beb"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h6n0"
server_key: "b9a0fdce9ae0a83eab715549a1e7db46"
server_key_iv: "44339fa7297653cb25c3aa712ec796ca"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:02.722258 27574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "296db31b064f47d39f52926ebef10beb"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h6n0"
server_key: "b9a0fdce9ae0a83eab715549a1e7db46"
server_key_iv: "44339fa7297653cb25c3aa712ec796ca"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:02.724335 27574 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:39:02.742296 27604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.742673 27574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:39:02.742787 27574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "296db31b064f47d39f52926ebef10beb"
format_stamp: "Formatted at 2026-08-07 08:39:02 on dist-test-slave-h6n0"
server_key: "b9a0fdce9ae0a83eab715549a1e7db46"
server_key_iv: "44339fa7297653cb25c3aa712ec796ca"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:02.742884 27574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:02.765658 27574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:02.768919 27574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:02.769248 27574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:02.769711 27574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:02.770298 27574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:02.770358 27574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.770395 27574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:02.770414 27574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:02.785624 27574 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:43367
I20260807 08:39:02.785804 27789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:43367 every 8 connection(s)
I20260807 08:39:02.786427 27574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:39:02.792984 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 27574
I20260807 08:39:02.793107 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:39:02.793332 11068 external_mini_cluster.cc:1428] Setting key 938ad7e4b0ca8214815b7f638bcdf16c
Aug 07 08:39:02 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091942, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:02.801379 27797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.787650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43615 (local address 127.10.207.2:56947)
0807 08:39:02.787961 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:02.788022 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:02.788981 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:02.789371 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:02.789376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:02.789735 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:02.790100 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.790107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.791329 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.791332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.792832 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:02.792837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.793415 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.793416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:02.793606 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.794356 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:02.794368 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:02.796349 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:02.798677 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.798684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.798698 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.799084 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.799356 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:02.799357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:02.799357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:02.799497 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:02.799855 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:02.799863 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:02.800068 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 08:39:02.800428 (+ 360us) client_negotiation.cc:240] Negotiation successful
0807 08:39:02.800825 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":185,"threads_started":1}
I20260807 08:39:02.801852 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:02.787658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:56947 (local address 127.10.207.62:43615)
0807 08:39:02.787737 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:02.787740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:02.788212 (+ 472us) server_negotiation.cc:407] Connection header received
0807 08:39:02.789138 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:02.789139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:02.789163 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:02.789266 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:02.790247 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.791141 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.792945 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:02.793265 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:02.793339 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:02.796469 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:02.796471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:02.796471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:02.796478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:02.798502 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.799196 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.799197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.799197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.799275 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:02.799582 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:02.799583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:02.799583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:02.799751 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:02.799824 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:02.801645 (+ 1821us) server_negotiation.cc:299] Negotiation successful
0807 08:39:02.801757 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:39:02.802614 27791 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43615
I20260807 08:39:02.802804 27791 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:02.803068 27791 heartbeater.cc:511] Master 127.10.207.62:43615 requested a full tablet report, sending...
I20260807 08:39:02.803663 27217 ts_manager.cc:194] Registered new tserver with Master: 296db31b064f47d39f52926ebef10beb (127.10.207.2:43367)
I20260807 08:39:02.804164 27217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:56947
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:39:02.834393 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:45829/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43615
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:03.018567 27800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:03.019030 27800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:03.019165 27800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:03.021569 27800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:03.021704 27800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:03.021828 27800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:39:03.024889 27800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.207.20:45829/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:03.025316 27800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:03.025771 27800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:03.034664 27810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.034926 27808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:03.035802 27813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:03.036273 27800 server_base.cc:1034] running on GCE node
I20260807 08:39:03.036574 27800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:03.036896 27800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:03.038087 27800 hybrid_clock.cc:648] HybridClock initialized: now 1786091943038064 us; error 70 us; skew 500 ppm
Aug 07 08:39:03 dist-test-slave-h6n0 krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091943, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:03.042827 27800 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:39:03.043473 27800 webserver.cc:460] Webserver started at http://127.10.207.3:46557/ using document root <none> and password file <none>
I20260807 08:39:03.043812 27800 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:03.043895 27800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:03.044112 27800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:03 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091943, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:03.062456 27800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "9420ac75e7a54e158207a92a8ba4af4d"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h6n0"
server_key: "372824829a7f305fc051b39ad53cb391"
server_key_iv: "39c7341b02ca580c499a5337bd1034dc"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:03.063218 27800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "9420ac75e7a54e158207a92a8ba4af4d"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h6n0"
server_key: "372824829a7f305fc051b39ad53cb391"
server_key_iv: "39c7341b02ca580c499a5337bd1034dc"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:03.067281 27800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:39:03.086845 27825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.087265 27800 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260807 08:39:03.087432 27800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "9420ac75e7a54e158207a92a8ba4af4d"
format_stamp: "Formatted at 2026-08-07 08:39:03 on dist-test-slave-h6n0"
server_key: "372824829a7f305fc051b39ad53cb391"
server_key_iv: "39c7341b02ca580c499a5337bd1034dc"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:03.087625 27800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:03.107549 27800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:03.110297 27800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:03.110564 27800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:03.110966 27800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:03.111467 27800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:03.111574 27800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.111658 27800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:03.111723 27800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:03.130671 27800 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:46523
I20260807 08:39:03.131233 27800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:39:03.136291 27999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:46523 every 8 connection(s)
I20260807 08:39:03.136768 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 27800
I20260807 08:39:03.136915 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:39:03.137172 11068 external_mini_cluster.cc:1428] Setting key 1d020ea8b0551a75ea7b99b0ff1699bb
Aug 07 08:39:03 dist-test-slave-h6n0 krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091943, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:03.152493 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.139519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:43617 (local address 127.10.207.62:43615)
0807 08:39:03.139567 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.139570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.139839 (+ 269us) server_negotiation.cc:407] Connection header received
0807 08:39:03.140854 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.140855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.140866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.140896 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:03.142031 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.142766 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.144383 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.144746 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.144865 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.148925 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:03.148927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:03.148927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:03.148937 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:03.150909 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.151375 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.151376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.151377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.151444 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.151732 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.151733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.151733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.151871 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:03.151945 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.152280 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.152384 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:39:03.152793 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.139098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43615 (local address 127.10.207.3:43617)
0807 08:39:03.139714 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.139757 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.140735 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.141071 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.141075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.141510 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:03.141922 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.141927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.142882 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.142883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.144229 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.144233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.144968 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.144970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.145229 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.146199 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:03.146211 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:03.148763 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:03.151002 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:03.151012 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:03.151015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:03.151286 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:03.151527 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:03.151528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:03.151529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:03.151653 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:03.151947 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:03.151956 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:03.152122 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.152243 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.152428 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":123,"threads_started":1}
I20260807 08:39:03.153481 28000 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43615
I20260807 08:39:03.153599 28000 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:03.153832 28000 heartbeater.cc:511] Master 127.10.207.62:43615 requested a full tablet report, sending...
I20260807 08:39:03.154276 27217 ts_manager.cc:194] Registered new tserver with Master: 9420ac75e7a54e158207a92a8ba4af4d (127.10.207.3:46523)
I20260807 08:39:03.154821 27217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:43617
I20260807 08:39:03.159976 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:39:03.171559 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.162140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53784 (local address 127.10.207.62:43615)
0807 08:39:03.162192 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.162194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.162678 (+ 484us) server_negotiation.cc:407] Connection header received
0807 08:39:03.162786 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.162787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.162799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.162831 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:03.163876 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.164614 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.165751 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.166045 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.166091 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.167877 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:03.167878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:03.167879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:03.167887 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:03.169994 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.170572 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.170573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.170574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.170643 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:03.170883 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:03.170884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:03.170885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:03.171036 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:03.171117 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.171348 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.171452 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:39:03.176268 27217 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:39:03.178511 27217 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:39:03.189160 28028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.182000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:46523 (local address 127.0.0.1:43260)
0807 08:39:03.183841 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.183851 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.183915 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.184285 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.184286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.184288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.184333 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.184334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.186063 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.186063 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.187198 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.187201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.188846 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.188847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.188934 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.188947 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.189012 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.189066 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1823,"mutex_wait_us":3,"thread_start_us":121,"threads_started":1}
I20260807 08:39:03.189891 28029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.182902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.10.207.3:46523)
0807 08:39:03.183787 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.183791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.183967 (+ 176us) server_negotiation.cc:407] Connection header received
0807 08:39:03.184010 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.184012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.184027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.184103 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.184416 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.185971 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.187366 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.188610 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.188684 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.188807 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.189721 (+ 914us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.189769 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":109,"threads_started":1}
I20260807 08:39:03.190963 27895 tablet_service.cc:1467] Processing CreateTablet for tablet 674a0c1540c84b2888b511b0f1f50807 (DEFAULT_TABLE table=test-table [id=1ab77cf3e9a644d7b2921d441d3406c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:03.191409 27895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 674a0c1540c84b2888b511b0f1f50807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:03.192616 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.182000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:40887 (local address 127.0.0.1:53224)
0807 08:39:03.182648 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.182699 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.182785 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.183212 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.183216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.183219 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.183342 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.183346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.185567 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.185568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.187414 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.187416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.192224 (+ 4808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.192227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.192343 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.192352 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.192445 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.192515 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":133,"threads_started":1}
I20260807 08:39:03.195390 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.182351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.10.207.1:40887)
0807 08:39:03.182942 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.182948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.182960 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:39:03.183005 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.183008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.183026 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.183116 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.183429 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.185480 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.189837 (+ 4357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.190658 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.190730 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.190786 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.195026 (+ 4240us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.195112 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":100,"threads_started":1}
I20260807 08:39:03.196096 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.182300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43367 (local address 127.0.0.1:34926)
0807 08:39:03.186641 (+ 4341us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.186653 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.186717 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.187833 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.187833 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.187835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.187876 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.187877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.190822 (+ 2945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.190824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.193348 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.193352 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.195828 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.195830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.195894 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.195899 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.195966 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.196017 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4314,"thread_start_us":153,"threads_started":1}
I20260807 08:39:03.196602 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.183113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.10.207.2:43367)
0807 08:39:03.183697 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.183701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.187025 (+ 3324us) server_negotiation.cc:407] Connection header received
0807 08:39:03.187585 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.187588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.187606 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.187693 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.187956 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.189748 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.193560 (+ 3812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.194733 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.194835 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.194912 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.196386 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.196432 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":116,"threads_started":1}
I20260807 08:39:03.197788 28034 tablet_bootstrap.cc:492] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Bootstrap starting.
I20260807 08:39:03.197837 27459 tablet_service.cc:1467] Processing CreateTablet for tablet 674a0c1540c84b2888b511b0f1f50807 (DEFAULT_TABLE table=test-table [id=1ab77cf3e9a644d7b2921d441d3406c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:03.198287 27459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 674a0c1540c84b2888b511b0f1f50807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:03.199859 28034 tablet_bootstrap.cc:654] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:03.200402 28034 log.cc:824] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:03.201623 28034 tablet_bootstrap.cc:492] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: No bootstrap required, opened a new log
I20260807 08:39:03.201701 28034 ts_tablet_manager.cc:1397] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:39:03.204188 27685 tablet_service.cc:1467] Processing CreateTablet for tablet 674a0c1540c84b2888b511b0f1f50807 (DEFAULT_TABLE table=test-table [id=1ab77cf3e9a644d7b2921d441d3406c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:03.204704 27685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 674a0c1540c84b2888b511b0f1f50807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:03.205542 28034 raft_consensus.cc:348] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.205758 28034 raft_consensus.cc:374] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:03.205802 28034 raft_consensus.cc:729] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9420ac75e7a54e158207a92a8ba4af4d, State: Initialized, Role: FOLLOWER
I20260807 08:39:03.206393 28034 consensus_queue.cc:260] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.207051 28034 ts_tablet_manager.cc:1428] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260807 08:39:03.207197 28000 heartbeater.cc:503] Master 127.10.207.62:43615 was elected leader, sending a full tablet report...
I20260807 08:39:03.209784 28041 tablet_bootstrap.cc:492] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: Bootstrap starting.
I20260807 08:39:03.210214 28039 tablet_bootstrap.cc:492] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169: Bootstrap starting.
I20260807 08:39:03.211845 28041 tablet_bootstrap.cc:654] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:03.212060 28039 tablet_bootstrap.cc:654] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:03.212519 28041 log.cc:824] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:03.212817 28039 log.cc:824] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:03.213763 28041 tablet_bootstrap.cc:492] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: No bootstrap required, opened a new log
I20260807 08:39:03.213845 28041 ts_tablet_manager.cc:1397] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:39:03.213874 28039 tablet_bootstrap.cc:492] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169: No bootstrap required, opened a new log
I20260807 08:39:03.213943 28039 ts_tablet_manager.cc:1397] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:39:03.218468 28039 raft_consensus.cc:348] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.218852 28039 raft_consensus.cc:374] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:03.218950 28039 raft_consensus.cc:729] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649b7ad34be24d24bbd991901c4a0169, State: Initialized, Role: FOLLOWER
I20260807 08:39:03.219121 28039 consensus_queue.cc:260] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.219370 28041 raft_consensus.cc:348] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.219570 28041 raft_consensus.cc:374] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:03.219628 28041 raft_consensus.cc:729] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 296db31b064f47d39f52926ebef10beb, State: Initialized, Role: FOLLOWER
I20260807 08:39:03.219545 28039 ts_tablet_manager.cc:1428] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:39:03.219883 27567 heartbeater.cc:503] Master 127.10.207.62:43615 was elected leader, sending a full tablet report...
I20260807 08:39:03.220233 28041 consensus_queue.cc:260] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.220597 28041 ts_tablet_manager.cc:1428] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260807 08:39:03.220804 27791 heartbeater.cc:503] Master 127.10.207.62:43615 was elected leader, sending a full tablet report...
I20260807 08:39:03.221966 28044 raft_consensus.cc:482] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:39:03.222165 28044 raft_consensus.cc:504] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.222499 28044 leader_election.cc:288] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9420ac75e7a54e158207a92a8ba4af4d (127.10.207.3:46523), 296db31b064f47d39f52926ebef10beb (127.10.207.2:43367)
I20260807 08:39:03.228735 28029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.223482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:33563 (local address 127.10.207.3:46523)
0807 08:39:03.223538 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.223541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.223836 (+ 295us) server_negotiation.cc:407] Connection header received
0807 08:39:03.223960 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.223962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.223973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.224005 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.224433 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.225264 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.227021 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.227708 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.227765 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.227777 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.228597 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.228656 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:39:03.229339 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.222794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:46523 (local address 127.10.207.1:33563)
0807 08:39:03.223714 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.223727 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.223873 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.224080 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.224082 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.224153 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.224327 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.224330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.225432 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.225433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.226909 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.226912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.228133 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.228134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.228276 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.228282 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.229136 (+ 854us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.229191 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":158,"threads_started":1}
I20260807 08:39:03.230325 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674a0c1540c84b2888b511b0f1f50807" candidate_uuid: "649b7ad34be24d24bbd991901c4a0169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420ac75e7a54e158207a92a8ba4af4d" is_pre_election: true
I20260807 08:39:03.230629 27912 raft_consensus.cc:2454] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 649b7ad34be24d24bbd991901c4a0169 in term 0.
I20260807 08:39:03.231135 27378 leader_election.cc:302] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 649b7ad34be24d24bbd991901c4a0169, 9420ac75e7a54e158207a92a8ba4af4d; no voters:
I20260807 08:39:03.231371 28044 raft_consensus.cc:2781] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:39:03.231487 28044 raft_consensus.cc:482] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:39:03.231590 28044 raft_consensus.cc:3037] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:03.233150 28044 raft_consensus.cc:504] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.233526 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674a0c1540c84b2888b511b0f1f50807" candidate_uuid: "649b7ad34be24d24bbd991901c4a0169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420ac75e7a54e158207a92a8ba4af4d"
I20260807 08:39:03.233623 27912 raft_consensus.cc:3037] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:03.234867 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.223423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:46499 (local address 127.10.207.2:43367)
0807 08:39:03.223466 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.223468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.228531 (+ 5063us) server_negotiation.cc:407] Connection header received
0807 08:39:03.229715 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.229717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.229732 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.229823 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:03.230131 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.231183 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.233524 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.234241 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.234305 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.234317 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.234605 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.234790 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:39:03.235110 27912 raft_consensus.cc:2454] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 649b7ad34be24d24bbd991901c4a0169 in term 1.
I20260807 08:39:03.235381 27378 leader_election.cc:302] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 649b7ad34be24d24bbd991901c4a0169, 9420ac75e7a54e158207a92a8ba4af4d; no voters:
I20260807 08:39:03.235569 28044 leader_election.cc:288] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [CANDIDATE]: Term 1 election: Requested vote from peers 9420ac75e7a54e158207a92a8ba4af4d (127.10.207.3:46523), 296db31b064f47d39f52926ebef10beb (127.10.207.2:43367)
I20260807 08:39:03.235678 28044 raft_consensus.cc:2781] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:03.235769 28044 raft_consensus.cc:686] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [term 1 LEADER]: Becoming Leader. State: Replica: 649b7ad34be24d24bbd991901c4a0169, State: Running, Role: LEADER
I20260807 08:39:03.235926 28044 consensus_queue.cc:237] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } }
I20260807 08:39:03.235694 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.223064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43367 (local address 127.10.207.1:46499)
0807 08:39:03.228454 (+ 5390us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.228466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:03.229584 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:03.229893 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:03.229895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:03.229898 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:03.230028 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.230031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.231467 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.231468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.233402 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:03.233405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.234391 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.234392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:03.234492 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.234497 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:39:03.235225 (+ 728us) client_negotiation.cc:240] Negotiation successful
0807 08:39:03.235286 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5344,"thread_start_us":147,"threads_started":1}
I20260807 08:39:03.237191 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674a0c1540c84b2888b511b0f1f50807" candidate_uuid: "649b7ad34be24d24bbd991901c4a0169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "296db31b064f47d39f52926ebef10beb" is_pre_election: true
I20260807 08:39:03.237190 27216 catalog_manager.cc:5495] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 reported cstate change: term changed from 0 to 1, leader changed from <none> to 649b7ad34be24d24bbd991901c4a0169 (127.10.207.1). New cstate: current_term: 1 leader_uuid: "649b7ad34be24d24bbd991901c4a0169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "649b7ad34be24d24bbd991901c4a0169" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 } health_report { overall_health: UNKNOWN } } }
I20260807 08:39:03.237366 27718 raft_consensus.cc:2454] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 649b7ad34be24d24bbd991901c4a0169 in term 0.
I20260807 08:39:03.237562 27716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "674a0c1540c84b2888b511b0f1f50807" candidate_uuid: "649b7ad34be24d24bbd991901c4a0169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "296db31b064f47d39f52926ebef10beb"
I20260807 08:39:03.237682 27716 raft_consensus.cc:3037] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:03.239310 27716 raft_consensus.cc:2454] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 649b7ad34be24d24bbd991901c4a0169 in term 1.
I20260807 08:39:03.256966 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.253194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.10.207.1:40887)
0807 08:39:03.253248 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:03.253250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:03.253258 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:39:03.253309 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:03.253310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:03.253335 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:03.253413 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:03.253650 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.254399 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.255633 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:03.255926 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:03.256001 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:03.256244 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:03.256411 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:03.256503 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:03.256784 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 08:39:03.256812 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:39:03.260228 27716 raft_consensus.cc:1261] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 1 FOLLOWER]: Refusing update from remote peer 649b7ad34be24d24bbd991901c4a0169: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:03.260666 28049 consensus_queue.cc:1035] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "296db31b064f47d39f52926ebef10beb" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:03.262985 27912 raft_consensus.cc:1261] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 1 FOLLOWER]: Refusing update from remote peer 649b7ad34be24d24bbd991901c4a0169: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:03.263285 28049 consensus_queue.cc:1035] T 674a0c1540c84b2888b511b0f1f50807 P 649b7ad34be24d24bbd991901c4a0169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9420ac75e7a54e158207a92a8ba4af4d" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:03.268147 28053 mvcc.cc:205] Tried to move back new op lower bound from 7315832599588724736 to 7315832599495659520. Current Snapshot: MvccSnapshot[applied={T|T < 7315832599588724736}]
I20260807 08:39:03.268707 28054 mvcc.cc:205] Tried to move back new op lower bound from 7315832599588724736 to 7315832599495659520. Current Snapshot: MvccSnapshot[applied={T|T < 7315832599588724736}]
I20260807 08:39:03.273190 28057 mvcc.cc:205] Tried to move back new op lower bound from 7315832599588724736 to 7315832599495659520. Current Snapshot: MvccSnapshot[applied={T|T < 7315832599588724736}]
I20260807 08:39:03.275297 27217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:03.275463 27217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:03.276680 27217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b1ced7ff51b4e8aa539f984be1b852b: Sending DeleteTablet for 3 replicas of tablet 674a0c1540c84b2888b511b0f1f50807
I20260807 08:39:03.277318 27895 tablet_service.cc:1514] Processing DeleteTablet for tablet 674a0c1540c84b2888b511b0f1f50807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:03 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:43260
I20260807 08:39:03.277783 27685 tablet_service.cc:1514] Processing DeleteTablet for tablet 674a0c1540c84b2888b511b0f1f50807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:03 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:34926
I20260807 08:39:03.278396 27459 tablet_service.cc:1514] Processing DeleteTablet for tablet 674a0c1540c84b2888b511b0f1f50807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:03 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:53224
I20260807 08:39:03.279260 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 27349
I20260807 08:39:03.291718 28070 tablet_replica.cc:330] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: stopping tablet replica
I20260807 08:39:03.292127 28075 tablet_replica.cc:330] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: stopping tablet replica
I20260807 08:39:03.292367 28075 raft_consensus.cc:2229] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:39:03.292631 28070 raft_consensus.cc:2229] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:39:03.292928 28075 raft_consensus.cc:2258] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:03.293087 28070 raft_consensus.cc:2258] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:03.293879 28075 ts_tablet_manager.cc:1905] T 674a0c1540c84b2888b511b0f1f50807 P 296db31b064f47d39f52926ebef10beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:03.294289 28070 ts_tablet_manager.cc:1905] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:39:03.295866 27192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:40887 (error 108)
I20260807 08:39:03.296487 28070 ts_tablet_manager.cc:1918] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:03.296579 28070 log.cc:1196] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/674a0c1540c84b2888b511b0f1f50807
I20260807 08:39:03.297158 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 27574
I20260807 08:39:03.297273 28070 ts_tablet_manager.cc:1939] T 674a0c1540c84b2888b511b0f1f50807 P 9420ac75e7a54e158207a92a8ba4af4d: Deleting consensus metadata
I20260807 08:39:03.303944 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.297332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:40887 (local address 127.0.0.1:53252)
0807 08:39:03.303811 (+ 6479us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.303882 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6439}
W20260807 08:39:03.304379 27192 catalog_manager.cc:4584] TS 649b7ad34be24d24bbd991901c4a0169 (127.10.207.1:40887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 674a0c1540c84b2888b511b0f1f50807: Network error: Client connection negotiation failed: client connection to 127.10.207.1:40887: connect: Connection refused (error 111)
I20260807 08:39:03.309563 27193 catalog_manager.cc:4841] TS 9420ac75e7a54e158207a92a8ba4af4d (127.10.207.3:46523): tablet 674a0c1540c84b2888b511b0f1f50807 (table test-table [id=1ab77cf3e9a644d7b2921d441d3406c6]) successfully deleted
I20260807 08:39:03.318388 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 27800
I20260807 08:39:03.324086 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:03.323945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43367 (local address 127.0.0.1:34930)
0807 08:39:03.324006 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:03.324036 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.2:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 08:39:03.333577 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 27168
I20260807 08:39:03.347519 11068 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:39:03.693693 11068 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:39:03.693830 11068 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:39:04.049044 11068 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:39:04.049 UTC [26536] LOG: received smart shutdown request
2026-08-07 08:39:04.052 UTC [26536] LOG: background worker "logical replication launcher" (PID 26549) exited with exit code 1
2026-08-07 08:39:04.053 UTC [26543] LOG: shutting down
2026-08-07 08:39:04.221 UTC [26536] LOG: database system is shut down
2026-08-07T08:39:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51051 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:04 dist-test-slave-h6n0 krb5kdc[28095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:04 dist-test-slave-h6n0 krb5kdc[28095](info): set up 2 sockets
Aug 07 08:39:04 dist-test-slave-h6n0 krb5kdc[28095](info): commencing operation
krb5kdc: starting...
W20260807 08:39:05.649286 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:39:05 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/rangeradmin_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/rangeradmin_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/rangerlookup_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/rangerlookup_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
I20260807 08:39:05.838744 11068 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDqD6RB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/postgres -l logfile start
2026-08-07 08:39:08.330 UTC [28146] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:39:08.330 UTC [28146] LOG: listening on IPv4 address "127.10.207.20", port 58719
2026-08-07 08:39:08.333 UTC [28146] LOG: listening on Unix socket "/tmp/.s.PGSQL.58719"
2026-08-07 08:39:08.357 UTC [28149] LOG: database system was shut down at 2026-08-07 08:39:06 UTC
2026-08-07 08:39:08.362 UTC [28146] LOG: database system is ready to accept connections
I20260807 08:39:09.536954 11068 mini_postgres.cc:96] Postgres bound to 58719
2026-08-07 08:39:09.544 UTC [28166] FATAL: database "slave" does not exist
127.10.207.20:58719 - accepting connections
I20260807 08:39:09.545585 11068 mini_ranger.cc:161] Starting Ranger...
I20260807 08:39:09.562685 11068 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:39:10.020169 11068 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:39:10.020277 11068 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-admin
2026-08-07 08:39:10,321 [I] DB FLAVOR :POSTGRES
2026-08-07 08:39:10,321 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:39:10,322 [I] Checking connection..
2026-08-07 08:39:10,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:39:10,620 [I] Checking connection passed.
2026-08-07 08:39:10,620 [I] --------- Verifying version history table ---------
2026-08-07 08:39:10,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:39:10,911 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:39:10,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:39:11,172 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:39:11,173 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:39:11,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:39:11.451 UTC [28270] WARNING: there is no transaction in progress
2026-08-07 08:39:11,461 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:39:11,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:39:11,801 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:39:11,801 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:39:11,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:39:12,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:39:12,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-07 08:39:12,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:39:12,865 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:39:12,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:39:13,135 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:39:13,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:39:13,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:39:13,696 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:39:13,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:39:14.671 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.687 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.704 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.719 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.734 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.751 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.809 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.816 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.830 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.841 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.851 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.867 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.877 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.883 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.890 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:14.897 UTC [28446] WARNING: there is no transaction in progress
2026-08-07 08:39:15,526 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:39:15,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:39:15,799 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:39:15,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:39:16,050 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:39:16,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:39:16,342 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:39:16,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:39:16,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:39:16,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-07 08:39:17,130 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:39:17,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:39:17,401 [I] DB_PATCHES have already been applied
I20260807 08:39:17.406997 11068 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDqD6RB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:39:17.407099 11068 mini_ranger.cc:193] Using host: 127.10.207.20
I20260807 08:39:17.411847 11068 mini_ranger.cc:227] Ranger admin URL: http://127.10.207.20:48647
Aug 07, 2026 8:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:39:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:39:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:39:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:39:18.823657 11068 mini_ranger.cc:160] Time spent starting Ranger: real 9.278s user 0.001s sys 0.011s
Aug 07, 2026 8:39:40 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:39:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:39:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:39:40 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:39:41 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:39:42.248001 11068 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/rangerkms_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/rangerkms_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.207.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.207.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.207.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/HTTP_127.10.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:39:42 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:39:42.352355 11068 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:39:42.363711 11068 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:39:43.195030 11068 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:39:43.195132 11068 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms
I20260807 08:39:43.195156 11068 mini_ranger_kms.cc:227] Using postgres at 127.10.207.20:58719
2026-08-07 08:39:43,417 [I] DB FLAVOR :POSTGRES
2026-08-07 08:39:43,417 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:39:43,418 [I] Checking connection
2026-08-07 08:39:43,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:39:43,737 [I] connection success
2026-08-07 08:39:43,737 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:39:43,737 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:39:43,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:39:44,002 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:39:44,002 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:39:44,002 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:39:44,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.207.20:58719/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:39:44,292 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:39:44.832870 11068 mini_ranger_kms.cc:310] Created kms service
I20260807 08:39:45.057323 11068 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:39:45.108302 11068 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:39:45 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091982, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:45.575410 11068 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:39:45.575691 11068 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDqD6RB/build/release/bin/postgresql.jar:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDqD6RB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:39:45.575716 11068 mini_ranger_kms.cc:240] Using host: 127.10.207.20
I20260807 08:39:45.580988 11068 mini_ranger_kms.cc:276] Ranger KMS PID: 28746
I20260807 08:39:45.581105 11068 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.207.20:44165
Aug 07, 2026 8:39:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:39:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:39:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:39:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 08:39:48 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091988, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.207.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:48.379328 11068 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.027s user 0.003s sys 0.204s
I20260807 08:39:48.379457 11068 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:39:48.379516 11068 mini_ranger_kms.cc:401] 127.10.207.20:44165/kms/v1/keys
Aug 07 08:39:48 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091988, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.207.20@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
Aug 07 08:39:48 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091988, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.207.20@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
Aug 07 08:39:51 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:39:51.140882 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:34327
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:34327
--ranger_config_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:51.235028 28829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:51.235308 28829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:51.235344 28829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:51.236563 28829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:51.236635 28829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:51.236661 28829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:51.236683 28829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:51.236712 28829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:51.238451 28829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.207.20:44165/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:34327
--ranger_config_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:34327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:51.238773 28829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:51.239034 28829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:51.244998 28836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:51.244980 28839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:51.245016 28829 server_base.cc:1034] running on GCE node
W20260807 08:39:51.244980 28841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:51.245639 28829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:51.246009 28829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:51.247216 28829 hybrid_clock.cc:648] HybridClock initialized: now 1786091991247196 us; error 40 us; skew 500 ppm
Aug 07 08:39:51 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091991, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:51.254408 28829 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:39:51.255018 28829 webserver.cc:460] Webserver started at http://127.10.207.62:46009/ using document root <none> and password file <none>
I20260807 08:39:51.255342 28829 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:51.255407 28829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:51.255573 28829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:51 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091991, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:51.312573 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "a1af7da3c5374088b826cc3497c12b27"
format_stamp: "Formatted at 2026-08-07 08:39:51 on dist-test-slave-h6n0"
server_key: "fc08233875f64f548eaf05cb66f6af08"
server_key_iv: "10c3d396b57b017f70c4ee4cdc8112b2"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:51.313120 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "a1af7da3c5374088b826cc3497c12b27"
format_stamp: "Formatted at 2026-08-07 08:39:51 on dist-test-slave-h6n0"
server_key: "fc08233875f64f548eaf05cb66f6af08"
server_key_iv: "10c3d396b57b017f70c4ee4cdc8112b2"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:51.315002 28829 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260807 08:39:51.332769 28850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:51.332995 28829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:39:51.333122 28829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "a1af7da3c5374088b826cc3497c12b27"
format_stamp: "Formatted at 2026-08-07 08:39:51 on dist-test-slave-h6n0"
server_key: "fc08233875f64f548eaf05cb66f6af08"
server_key_iv: "10c3d396b57b017f70c4ee4cdc8112b2"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:51.333252 28829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:51.358999 28829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:51.361330 28829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:51.361529 28829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:51.367022 28829 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:34327
I20260807 08:39:51.367093 28929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:34327 every 8 connection(s)
I20260807 08:39:51.367475 28829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:51.368364 28930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:51.368525 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 28829
I20260807 08:39:51.368673 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:39:51.368912 11068 external_mini_cluster.cc:1428] Setting key d62209125fdc657ea4852fe14cdc8522
I20260807 08:39:51.371702 28930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Bootstrap starting.
I20260807 08:39:51.372444 28930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:51.372844 28930 log.cc:824] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:51.373638 28930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: No bootstrap required, opened a new log
I20260807 08:39:51.375746 28930 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
I20260807 08:39:51.375907 28930 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:51.375936 28930 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1af7da3c5374088b826cc3497c12b27, State: Initialized, Role: FOLLOWER
I20260807 08:39:51.376083 28930 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
I20260807 08:39:51.376173 28930 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:51.376204 28930 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:51.376279 28930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:51.377081 28930 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
Aug 07 08:39:51 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:51.377225 28930 leader_election.cc:302] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1af7da3c5374088b826cc3497c12b27; no voters:
I20260807 08:39:51.377413 28930 leader_election.cc:288] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:39:51.377449 28938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:51.377590 28938 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 LEADER]: Becoming Leader. State: Replica: a1af7da3c5374088b826cc3497c12b27, State: Running, Role: LEADER
I20260807 08:39:51.377749 28930 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:51.377779 28938 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
I20260807 08:39:51.378314 28941 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1af7da3c5374088b826cc3497c12b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } } }
I20260807 08:39:51.378451 28941 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:51.378865 28949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:51.379079 28942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1af7da3c5374088b826cc3497c12b27. Latest consensus state: current_term: 1 leader_uuid: "a1af7da3c5374088b826cc3497c12b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } } }
I20260807 08:39:51.379170 28942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:51.379503 28949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:51.379858 28930 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:39:51.381546 28949 catalog_manager.cc:1347] Generated new cluster ID: 9998245925b64ddcbd3baf574c28921d
I20260807 08:39:51.381609 28949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:51.382315 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:51.369592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.10.207.62:34327)
0807 08:39:51.369865 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:51.369877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:51.369912 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:39:51.370623 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:51.370635 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:51.370702 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:51.370879 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:51.371749 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:51.373078 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:51.374077 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:51.374628 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:51.374781 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:51.377907 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:51.377911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:51.377912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:51.377924 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:51.380251 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:51.380669 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:51.380673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:51.380673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:51.380802 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:51.381168 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:51.381169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:51.381170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:51.381500 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:51.381670 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:51.381934 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 08:39:51.382071 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":123,"threads_started":1}
I20260807 08:39:51.430893 28949 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:51.431951 28949 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:51.435985 28949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Generated new TSK 0
I20260807 08:39:51.436201 28949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:39:52.198979 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34327
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:52.321316 29003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:52.321547 29003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:52.321578 29003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:52.322885 29003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:52.322929 29003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:52.322956 29003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:39:52.324510 29003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.207.20:44165/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:52.324769 29003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:52.325042 29003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:52.330806 29009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:52.330806 29008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:52.331090 29003 server_base.cc:1034] running on GCE node
W20260807 08:39:52.331291 29014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:52.331571 29003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:52.331857 29003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:52.333046 29003 hybrid_clock.cc:648] HybridClock initialized: now 1786091992333011 us; error 50 us; skew 500 ppm
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:52.336696 29003 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:39:52.337296 29003 webserver.cc:460] Webserver started at http://127.10.207.1:37561/ using document root <none> and password file <none>
I20260807 08:39:52.337567 29003 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:52.337636 29003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:52.337808 29003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:52.354260 29003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "5475b36663574de6bb7d586c8bf3accf"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "e3f363cbbf6aaa3679bd1a6b25cd5288"
server_key_iv: "ac6f0dc288f4cdd52e5d6c897e9add95"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.354775 29003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "5475b36663574de6bb7d586c8bf3accf"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "e3f363cbbf6aaa3679bd1a6b25cd5288"
server_key_iv: "ac6f0dc288f4cdd52e5d6c897e9add95"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.356808 29003 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:39:52.377075 29026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.377381 29003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:39:52.377549 29003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "5475b36663574de6bb7d586c8bf3accf"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "e3f363cbbf6aaa3679bd1a6b25cd5288"
server_key_iv: "ac6f0dc288f4cdd52e5d6c897e9add95"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.377681 29003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:52.392688 29003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:52.395383 29003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:52.395587 29003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:52.395920 29003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:52.396360 29003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:52.396416 29003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.396451 29003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:52.396471 29003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.405874 29003 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:41903
I20260807 08:39:52.406412 29003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:39:52.407003 29198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:41903 every 8 connection(s)
I20260807 08:39:52.408306 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 29003
I20260807 08:39:52.408444 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:39:52.408664 11068 external_mini_cluster.cc:1428] Setting key c9d949e19540801c539730410fe778a2
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:52.423169 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:52.408063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:57907 (local address 127.10.207.62:34327)
0807 08:39:52.408270 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:52.408274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:52.408979 (+ 705us) server_negotiation.cc:407] Connection header received
0807 08:39:52.409693 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:52.409694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:52.409711 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:52.409809 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:52.410703 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.411516 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.413285 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.413629 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.413709 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:52.419113 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:52.419116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:52.419116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:52.419128 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:52.421286 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:52.421872 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:52.421873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:52.421874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:52.421944 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:52.422248 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:52.422250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:52.422250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:52.422435 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:52.422487 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:52.422949 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 08:39:52.423054 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260807 08:39:52.423802 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:52.407991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34327 (local address 127.10.207.1:57907)
0807 08:39:52.408355 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:52.408411 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:52.409572 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:52.409910 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:52.409915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:52.410219 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:52.410588 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:52.410597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.411626 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.411628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:52.413201 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:52.413205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.413748 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.413749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:52.413912 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:52.416926 (+ 3014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:52.416939 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:52.418930 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:52.421406 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:52.421414 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:52.421417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:52.421768 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:52.422020 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:52.422021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:52.422021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:52.422165 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:52.422567 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:52.422574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:52.422763 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 08:39:52.423348 (+ 585us) client_negotiation.cc:240] Negotiation successful
0807 08:39:52.423549 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
I20260807 08:39:52.424795 29199 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34327
I20260807 08:39:52.424921 29199 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:52.425156 29199 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:52.425989 28881 ts_manager.cc:194] Registered new tserver with Master: 5475b36663574de6bb7d586c8bf3accf (127.10.207.1:41903)
I20260807 08:39:52.426934 28881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:57907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:39:52.445140 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34327
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:52.561829 29205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:52.562108 29205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:52.562150 29205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:52.563548 29205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:52.563635 29205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:52.563687 29205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:39:52.565425 29205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.207.20:44165/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:52.565852 29205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:52.566154 29205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 08:39:52.573405 29222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:52.573422 29218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:52.573848 29219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:52.574647 29205 server_base.cc:1034] running on GCE node
I20260807 08:39:52.574901 29205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:52.575223 29205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:52.576465 29205 hybrid_clock.cc:648] HybridClock initialized: now 1786091992576440 us; error 46 us; skew 500 ppm
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:52.579993 29205 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:39:52.580631 29205 webserver.cc:460] Webserver started at http://127.10.207.2:39361/ using document root <none> and password file <none>
I20260807 08:39:52.581003 29205 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:52.581125 29205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:52.581319 29205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:52.599953 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "adc06c11486b409785a372c55295aa43"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "460fc6b2860404ca55e27ee4d9dfcd5d"
server_key_iv: "2a63a80a77710d803d1a4a62d565c883"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.600505 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "adc06c11486b409785a372c55295aa43"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "460fc6b2860404ca55e27ee4d9dfcd5d"
server_key_iv: "2a63a80a77710d803d1a4a62d565c883"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.602255 29205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:39:52.630594 29232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.630890 29205 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 08:39:52.631013 29205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "adc06c11486b409785a372c55295aa43"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "460fc6b2860404ca55e27ee4d9dfcd5d"
server_key_iv: "2a63a80a77710d803d1a4a62d565c883"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.631156 29205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:52.644178 29205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:52.647838 29205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:52.648061 29205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:52.648376 29205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:52.648818 29205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:52.648913 29205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.648991 29205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:52.649057 29205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.659760 29205 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:44543
I20260807 08:39:52.660457 29205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:39:52.662580 29415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:44543 every 8 connection(s)
I20260807 08:39:52.668972 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 29205
I20260807 08:39:52.669104 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:39:52.669377 11068 external_mini_cluster.cc:1428] Setting key 6c25ec98ac2e2ee07fc854cef3f5e777
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:52.673691 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:52.662125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:40303 (local address 127.10.207.62:34327)
0807 08:39:52.662183 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:52.662186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:52.662591 (+ 405us) server_negotiation.cc:407] Connection header received
0807 08:39:52.663347 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:52.663348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:52.663365 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:52.663396 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:52.664246 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.664997 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.666692 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.666979 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.667109 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:52.670252 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:52.670254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:52.670254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:52.670261 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:52.672094 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:52.672616 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:52.672617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:52.672618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:52.672677 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:52.672950 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:52.672951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:52.672952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:52.673085 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:52.673129 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:52.673487 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 08:39:52.673590 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:39:52.674470 29423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:52.661777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34327 (local address 127.10.207.2:40303)
0807 08:39:52.662437 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:52.662481 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:52.663213 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:52.663587 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:52.663591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:52.663840 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:52.664127 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:52.664134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.665291 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.665292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:52.666581 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:52.666585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.667235 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.667236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:52.667395 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:52.668301 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:52.668311 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:52.670134 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:52.672195 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:52.672201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:52.672203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:52.672502 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:52.672748 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:52.672748 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:52.672748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:52.672877 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:52.673176 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:52.673183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:52.673336 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 08:39:52.674066 (+ 730us) client_negotiation.cc:240] Negotiation successful
0807 08:39:52.674228 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":97,"threads_started":1}
I20260807 08:39:52.675311 29419 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34327
I20260807 08:39:52.675405 29419 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:52.675598 29419 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:52.675980 28881 ts_manager.cc:194] Registered new tserver with Master: adc06c11486b409785a372c55295aa43 (127.10.207.2:44543)
I20260807 08:39:52.676445 28881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:40303
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:39:52.709523 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34327
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:52.842558 29428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:52.842896 29428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:52.843007 29428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:52.844720 29428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:52.845008 29428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:52.845149 29428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:39:52.847389 29428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.207.20:44165/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:52.847826 29428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:52.848191 29428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:52.855196 29437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:52.855984 29441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:52.856542 29428 server_base.cc:1034] running on GCE node
W20260807 08:39:52.856884 29436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:52.857275 29428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:52.857656 29428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:52.860913 29428 hybrid_clock.cc:648] HybridClock initialized: now 1786091992860887 us; error 38 us; skew 500 ppm
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:52.865564 29428 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:39:52.866379 29428 webserver.cc:460] Webserver started at http://127.10.207.3:46865/ using document root <none> and password file <none>
I20260807 08:39:52.866760 29428 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:52.866901 29428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:52.867141 29428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:52.884523 29428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "c883ab31e27e4fd6a11aa245f0ee146b"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "15027fe5cfc3654c2f03ad30203d3fb6"
server_key_iv: "17a4159fcd798ce462b1aef37cbcb3e1"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.885011 29428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "c883ab31e27e4fd6a11aa245f0ee146b"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "15027fe5cfc3654c2f03ad30203d3fb6"
server_key_iv: "17a4159fcd798ce462b1aef37cbcb3e1"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.887277 29428 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260807 08:39:52.909196 29454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.909641 29428 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260807 08:39:52.909824 29428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "c883ab31e27e4fd6a11aa245f0ee146b"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "15027fe5cfc3654c2f03ad30203d3fb6"
server_key_iv: "17a4159fcd798ce462b1aef37cbcb3e1"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:52.910034 29428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:52.933014 29428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:52.935971 29428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:52.936221 29428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:52.936630 29428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:52.937155 29428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:52.937252 29428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.937335 29428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:52.937398 29428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:52.945560 29428 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:33973
I20260807 08:39:52.946178 29428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:39:52.955080 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 29428
I20260807 08:39:52.955242 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:39:52.955525 11068 external_mini_cluster.cc:1428] Setting key 3f2855cfe5e94f660529871a0a17159c
I20260807 08:39:52.956543 29622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:33973 every 8 connection(s)
Aug 07 08:39:52 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091992, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:52.982650 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:52.957564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:36519 (local address 127.10.207.62:34327)
0807 08:39:52.957614 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:52.957617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:52.970018 (+ 12401us) server_negotiation.cc:407] Connection header received
0807 08:39:52.971022 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:52.971023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:52.971038 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:52.971073 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:52.972184 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.973024 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.974837 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.975162 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.975215 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:52.978995 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:52.978997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:52.978997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:52.979005 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:52.980810 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:52.981383 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:52.981385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:52.981385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:52.981449 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:52.981751 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:52.981752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:52.981752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:52.981916 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:52.981971 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:52.982423 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 08:39:52.982515 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:39:52.983384 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:52.957530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34327 (local address 127.10.207.3:36519)
0807 08:39:52.957800 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:52.957855 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:52.970866 (+ 13011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:52.971335 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:52.971341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:52.971686 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:52.972039 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:52.972046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.973166 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.973167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:52.974728 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:52.974732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:52.975352 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:52.975354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:52.975514 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:52.976532 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:52.976546 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:52.978889 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:52.980914 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:52.980922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:52.980926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:52.981269 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:52.981514 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:52.981515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:52.981515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:52.981672 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:52.982035 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:52.982042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:52.982235 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 08:39:52.982814 (+ 579us) client_negotiation.cc:240] Negotiation successful
0807 08:39:52.983012 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":140,"threads_started":1}
I20260807 08:39:52.984063 29623 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34327
I20260807 08:39:52.984247 29623 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:52.984556 29623 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:52.985138 28881 ts_manager.cc:194] Registered new tserver with Master: c883ab31e27e4fd6a11aa245f0ee146b (127.10.207.3:33973)
I20260807 08:39:52.985649 28881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:36519
I20260807 08:39:52.989640 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:39:52.990345 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 29003
I20260807 08:39:53.020493 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 29205
I20260807 08:39:53.072402 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 29428
I20260807 08:39:53.093704 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 28829
I20260807 08:39:53.106247 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:34327
--webserver_interface=127.10.207.62
--webserver_port=46009
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:34327
--ranger_config_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:53.241863 29626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:53.242128 29626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:53.242156 29626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:53.243577 29626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:53.243626 29626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:53.243642 29626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:53.243655 29626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:53.243669 29626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:53.245473 29626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:34327
--ranger_config_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:34327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=46009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:53.245779 29626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:53.246039 29626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:53.252584 29635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:53.252746 29638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:53.252559 29632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:53.253051 29626 server_base.cc:1034] running on GCE node
I20260807 08:39:53.253373 29626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:53.253720 29626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:53.254899 29626 hybrid_clock.cc:648] HybridClock initialized: now 1786091993254876 us; error 41 us; skew 500 ppm
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:53.258365 29626 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:39:53.258973 29626 webserver.cc:460] Webserver started at http://127.10.207.62:46009/ using document root <none> and password file <none>
I20260807 08:39:53.259232 29626 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:53.259310 29626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:53.338944 29626 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:39:53.339867 29648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.340039 29626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:39:53.340148 29626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "a1af7da3c5374088b826cc3497c12b27"
format_stamp: "Formatted at 2026-08-07 08:39:51 on dist-test-slave-h6n0"
server_key: "fc08233875f64f548eaf05cb66f6af08"
server_key_iv: "10c3d396b57b017f70c4ee4cdc8112b2"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:53.340654 29626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:53.358052 29626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:53.360541 29626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:53.360774 29626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:53.370260 29626 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:34327
I20260807 08:39:53.370355 29736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:34327 every 8 connection(s)
I20260807 08:39:53.370913 29626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:53.372455 29737 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:39:53.373248 29737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Bootstrap starting.
I20260807 08:39:53.376297 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 29626
I20260807 08:39:53.376713 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:41903
--local_ip_for_outbound_sockets=127.10.207.1
--tserver_master_addrs=127.10.207.62:34327
--webserver_port=37561
--webserver_interface=127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:39:53.404727 29737 log.cc:824] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:53.406833 29737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:39:53.407124 29737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Bootstrap complete.
I20260807 08:39:53.409543 29737 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
I20260807 08:39:53.409798 29737 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1af7da3c5374088b826cc3497c12b27, State: Initialized, Role: FOLLOWER
I20260807 08:39:53.409996 29737 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
I20260807 08:39:53.410094 29737 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:39:53.410147 29737 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:39:53.410195 29737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:39:53.411327 29737 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
I20260807 08:39:53.411448 29737 leader_election.cc:302] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1af7da3c5374088b826cc3497c12b27; no voters:
I20260807 08:39:53.411635 29737 leader_election.cc:288] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:39:53.411860 29746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:39:53.411972 29737 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:53.412184 29746 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [term 2 LEADER]: Becoming Leader. State: Replica: a1af7da3c5374088b826cc3497c12b27, State: Running, Role: LEADER
I20260807 08:39:53.412425 29746 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } }
I20260807 08:39:53.413102 29746 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1af7da3c5374088b826cc3497c12b27. Latest consensus state: current_term: 2 leader_uuid: "a1af7da3c5374088b826cc3497c12b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } } }
I20260807 08:39:53.413188 29746 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:53.413539 29750 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a1af7da3c5374088b826cc3497c12b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af7da3c5374088b826cc3497c12b27" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34327 } } }
I20260807 08:39:53.413640 29750 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:53.414145 29760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:53.414717 29760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:53.414862 29760 catalog_manager.cc:1259] Loaded cluster ID: 9998245925b64ddcbd3baf574c28921d
I20260807 08:39:53.414892 29760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:53.415082 29737 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:39:53.415889 29760 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:53.416038 29760 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Loaded TSK: 0
I20260807 08:39:53.416180 29760 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:39:53.534590 29743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:53.534900 29743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:53.534942 29743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:53.536872 29743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:53.536986 29743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:53.537060 29743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:39:53.539535 29743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:41903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=37561
--tserver_master_addrs=127.10.207.62:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:53.539844 29743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:53.540233 29743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:53.547631 29798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:53.547641 29796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:53.548101 29743 server_base.cc:1034] running on GCE node
W20260807 08:39:53.548084 29794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:53.548525 29743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:53.548873 29743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:53.550069 29743 hybrid_clock.cc:648] HybridClock initialized: now 1786091993550043 us; error 44 us; skew 500 ppm
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:53.553828 29743 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:39:53.554407 29743 webserver.cc:460] Webserver started at http://127.10.207.1:37561/ using document root <none> and password file <none>
I20260807 08:39:53.554764 29743 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:53.554883 29743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:53.575033 29743 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:39:53.576180 29811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.576416 29743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:39:53.576534 29743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "5475b36663574de6bb7d586c8bf3accf"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "e3f363cbbf6aaa3679bd1a6b25cd5288"
server_key_iv: "ac6f0dc288f4cdd52e5d6c897e9add95"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:53.577059 29743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:53.611331 29743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:53.614319 29743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:53.614589 29743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:53.614955 29743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:53.615406 29743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:53.615443 29743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.615469 29743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:53.615515 29743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.624945 29743 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:41903
I20260807 08:39:53.625177 29995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:41903 every 8 connection(s)
I20260807 08:39:53.625507 29743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:39:53.626122 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 29743
I20260807 08:39:53.626559 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:44543
--local_ip_for_outbound_sockets=127.10.207.2
--tserver_master_addrs=127.10.207.62:34327
--webserver_port=39361
--webserver_interface=127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:53.645645 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:53.629531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:41737 (local address 127.10.207.62:34327)
0807 08:39:53.629772 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:53.629783 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:53.631382 (+ 1599us) server_negotiation.cc:407] Connection header received
0807 08:39:53.631541 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:53.631547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:53.631612 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:53.631732 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:53.632623 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.634154 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.635914 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.636726 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.636899 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:53.641087 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:53.641093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:53.641093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:53.641113 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:53.643252 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:53.643828 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:53.643832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:53.643833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:53.643961 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:53.644250 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:53.644251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:53.644251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:53.644607 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:53.644695 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:53.645228 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 08:39:53.645378 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":88,"threads_started":1}
I20260807 08:39:53.646224 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:53.629454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34327 (local address 127.10.207.1:41737)
0807 08:39:53.630357 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:53.630405 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:53.631183 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:53.631894 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:53.631898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:53.632172 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:53.632474 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:53.632481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.634289 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.634290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:53.635775 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:53.635779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.636898 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.636900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:53.637071 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:53.638456 (+ 1385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:53.638468 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:53.640934 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:53.643377 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:53.643385 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:53.643388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:53.643720 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:53.644040 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:53.644041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:53.644041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:53.644176 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:53.644728 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:53.644735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:53.644894 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 08:39:53.645013 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 08:39:53.645995 (+ 982us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":150,"threads_started":1}
I20260807 08:39:53.647137 29996 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34327
I20260807 08:39:53.647259 29996 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:53.647495 29996 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
W20260807 08:39:53.648514 29996 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:34327 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:39:53.648613 29996 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:53.648689 29996 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:53.649056 29996 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:53.649174 29996 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
W20260807 08:39:53.649533 29996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:39:53.752241 29999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:53.752513 29999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:53.752547 29999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:53.754207 29999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:53.754266 29999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:53.754304 29999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:39:53.756284 29999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:44543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=39361
--tserver_master_addrs=127.10.207.62:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:53.756543 29999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:53.756767 29999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:53.763682 30010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:53.763944 30014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:53.764058 30008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:53.764285 29999 server_base.cc:1034] running on GCE node
I20260807 08:39:53.764595 29999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:53.764887 29999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:53.766079 29999 hybrid_clock.cc:648] HybridClock initialized: now 1786091993766067 us; error 33 us; skew 500 ppm
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:53.770282 29999 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:39:53.770881 29999 webserver.cc:460] Webserver started at http://127.10.207.2:39361/ using document root <none> and password file <none>
I20260807 08:39:53.771160 29999 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:53.771255 29999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:53.785962 29999 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:39:53.787252 30026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.787447 29999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:39:53.787580 29999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "adc06c11486b409785a372c55295aa43"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "460fc6b2860404ca55e27ee4d9dfcd5d"
server_key_iv: "2a63a80a77710d803d1a4a62d565c883"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:53.787981 29999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:53.801268 29999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:53.803484 29999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:53.803643 29999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:53.803889 29999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:53.804252 29999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:53.804304 29999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.804342 29999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:53.804368 29999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.812881 29999 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:44543
I20260807 08:39:53.813015 30196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:44543 every 8 connection(s)
I20260807 08:39:53.813398 29999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:39:53.813635 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 29999
I20260807 08:39:53.814002 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.207.20:44165/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:33973
--local_ip_for_outbound_sockets=127.10.207.3
--tserver_master_addrs=127.10.207.62:34327
--webserver_port=46865
--webserver_interface=127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:53.825757 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:53.814745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:34053 (local address 127.10.207.62:34327)
0807 08:39:53.814811 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:53.814813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:53.814864 (+ 51us) server_negotiation.cc:407] Connection header received
0807 08:39:53.815672 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:53.815675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:53.815695 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:53.815728 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:53.816426 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.817218 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.818478 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.818809 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.818875 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:53.821682 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:53.821685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:53.821686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:53.821699 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:53.823721 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:53.824313 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:53.824316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:53.824316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:53.824396 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:53.824752 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:53.824753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:53.824754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:53.824938 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:53.825020 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:53.825520 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 08:39:53.825646 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:39:53.826787 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:53.814364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34327 (local address 127.10.207.2:34053)
0807 08:39:53.814733 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:53.814777 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:53.815530 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:53.815925 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:53.815930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:53.816141 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:53.816328 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:53.816331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.817337 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.817338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:53.818388 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:53.818391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:53.818971 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:53.818972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:53.819108 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:53.819646 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:53.819657 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:53.821534 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:53.823851 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:53.823858 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:53.823861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:53.824205 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:53.824496 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:53.824497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:53.824498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:53.824652 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:53.825068 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:53.825081 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:53.825326 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 08:39:53.826227 (+ 901us) client_negotiation.cc:240] Negotiation successful
0807 08:39:53.826449 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":104,"threads_started":1}
I20260807 08:39:53.827800 30197 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34327
I20260807 08:39:53.827922 30197 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:53.828178 30197 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
W20260807 08:39:53.829118 30197 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:34327 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:39:53.829200 30197 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:53.829270 30197 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:53.829752 30197 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:53.829835 30197 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
W20260807 08:39:53.830257 30197 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:39:53.915469 30204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:53.915664 30204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:53.915688 30204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:53.916893 30204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:53.916939 30204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:53.916965 30204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:39:53.919025 30204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.207.20:44165/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:33973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=46865
--tserver_master_addrs=127.10.207.62:34327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:53.919245 30204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:53.919492 30204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:53.926587 30210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:53.926837 30211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:53.927045 30217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:53.927526 30204 server_base.cc:1034] running on GCE node
I20260807 08:39:53.927784 30204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:53.928120 30204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:53.943701 30204 hybrid_clock.cc:648] HybridClock initialized: now 1786091993943649 us; error 65 us; skew 500 ppm
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:53.946816 30204 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:39:53.947371 30204 webserver.cc:460] Webserver started at http://127.10.207.3:46865/ using document root <none> and password file <none>
I20260807 08:39:53.947624 30204 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:53.947703 30204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:39:53 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:53.970953 30204 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260807 08:39:53.971875 30229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.972298 30204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:39:53.972397 30204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "c883ab31e27e4fd6a11aa245f0ee146b"
format_stamp: "Formatted at 2026-08-07 08:39:52 on dist-test-slave-h6n0"
server_key: "15027fe5cfc3654c2f03ad30203d3fb6"
server_key_iv: "17a4159fcd798ce462b1aef37cbcb3e1"
server_key_version: "kuduclusterkey@0"
I20260807 08:39:53.972777 30204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:53.982913 30204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:53.985625 30204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:53.985785 30204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:53.986042 30204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:39:53.986411 30204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:39:53.986469 30204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.986507 30204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:39:53.986562 30204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:53.993391 30204 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:33973
I20260807 08:39:53.993831 30204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:39:53.995460 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 30204
I20260807 08:39:53.996387 30409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:33973 every 8 connection(s)
Aug 07 08:39:54 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:54.011226 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:53.996205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.10.207.62:34327)
0807 08:39:53.996313 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:53.996316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:54.002833 (+ 6517us) server_negotiation.cc:407] Connection header received
0807 08:39:54.002935 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:54.002936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:54.002949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:54.003013 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:54.003994 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:54.004539 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:54.005448 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:54.005738 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:54.005828 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:54.007700 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:54.007702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:54.007703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:54.007711 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:54.009543 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:54.010360 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:54.010362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:54.010362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:54.010423 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:54.010655 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:54.010656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:54.010656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:54.010768 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:54.010967 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:54.011016 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:39:54.011106 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:39:54.015173 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:53.996882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34327 (local address 127.10.207.3:45861)
0807 08:39:54.002677 (+ 5795us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:54.002716 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:54.003331 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:54.003608 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:54.003612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:54.003810 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:54.004044 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:54.004049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:54.005258 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:54.005259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:54.006318 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:54.006321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:54.007946 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:54.007947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:54.008087 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:54.008573 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:54.008584 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:54.010717 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:54.013344 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:54.013353 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:54.013358 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:54.013759 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:54.014046 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:54.014046 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:54.014047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:54.014210 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:54.014596 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:54.014603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:54.014783 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 08:39:54.014890 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 08:39:54.015012 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5745,"thread_start_us":109,"threads_started":1}
I20260807 08:39:54.015198 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:53.997177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:45861 (local address 127.10.207.62:34327)
0807 08:39:54.003089 (+ 5912us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:54.003090 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:54.003097 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:39:54.003434 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:54.003435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:54.003447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:54.003533 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:54.004184 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:54.005137 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:54.006403 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:54.007759 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:54.007849 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:54.011342 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:54.011343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:54.011344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:54.011351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:54.013226 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:54.013842 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:54.013843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:54.013844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:54.013924 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:54.014308 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:54.014309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:54.014309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:54.014473 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:54.014583 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:54.014959 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 08:39:54.015078 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5884,"thread_start_us":70,"threads_started":1}
I20260807 08:39:54.016031 30411 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34327
I20260807 08:39:54.016144 30411 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:54.016355 30411 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
W20260807 08:39:54.017222 30411 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:34327 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:39:54.017376 30411 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:54.017514 30411 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:54.017941 30411 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:54.018071 30411 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
W20260807 08:39:54.018577 30411 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 08:39:54 dist-test-slave-h6n0 krb5kdc[28095](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091994, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:54.649817 29996 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:54.650080 29996 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:54.650655 29675 ts_manager.cc:194] Registered new tserver with Master: 5475b36663574de6bb7d586c8bf3accf (127.10.207.1:41903)
I20260807 08:39:54.651688 29675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:41737
I20260807 08:39:54.830451 30197 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:54.830685 30197 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:54.831251 29675 ts_manager.cc:194] Registered new tserver with Master: adc06c11486b409785a372c55295aa43 (127.10.207.2:44543)
I20260807 08:39:54.831859 29675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:34053
I20260807 08:39:55.018779 30411 heartbeater.cc:464] Registering TS with master...
I20260807 08:39:55.018929 30411 heartbeater.cc:511] Master 127.10.207.62:34327 requested a full tablet report, sending...
I20260807 08:39:55.019318 29675 ts_manager.cc:194] Registered new tserver with Master: c883ab31e27e4fd6a11aa245f0ee146b (127.10.207.3:33973)
I20260807 08:39:55.019968 29675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:45861
I20260807 08:39:55.023108 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:39:55.033095 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.024230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.10.207.62:34327)
0807 08:39:55.024537 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:55.024540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:55.024734 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:39:55.024760 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:55.024761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:55.024774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:55.024838 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:55.025817 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.026639 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.027661 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.027972 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.028034 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.029550 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:55.029552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:55.029552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:55.029559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:55.031518 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:55.031921 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:55.031923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:55.031923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:55.031988 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:55.032194 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:55.032195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:55.032196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:55.032323 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:55.032416 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:55.032887 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 08:39:55.032969 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":91,"threads_started":1}
I20260807 08:39:55.035343 29675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
Aug 07 08:39:55 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091994, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
W20260807 08:39:55.038137 29675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:39:55.064500 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.059001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41903 (local address 127.0.0.1:45968)
0807 08:39:55.060009 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:55.060019 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:55.060067 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:55.060541 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:55.060542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:55.060546 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:55.060622 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.060625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.061865 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.061866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.063152 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.063155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.064183 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.064184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.064261 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.064274 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:39:55.064341 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:39:55.064401 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":53,"thread_start_us":137,"threads_started":1}
I20260807 08:39:55.064883 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.059584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.10.207.1:41903)
0807 08:39:55.059883 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:55.059887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:55.060251 (+ 364us) server_negotiation.cc:407] Connection header received
0807 08:39:55.060303 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:55.060306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:55.060325 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:55.060442 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:55.060702 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.061758 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.063264 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.064008 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.064081 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.064144 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:55.064753 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 08:39:55.064805 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":72,"threads_started":1}
I20260807 08:39:55.065600 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.058985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:33973 (local address 127.0.0.1:59114)
0807 08:39:55.059650 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:55.059691 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:55.059763 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:55.060454 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:55.060457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:55.060461 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:55.060534 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.060536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.061806 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.061807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.063532 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.063535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.065342 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.065343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.065412 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.065416 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:39:55.065463 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:39:55.065511 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":80,"threads_started":1}
I20260807 08:39:55.065714 29886 tablet_service.cc:1467] Processing CreateTablet for tablet 5831ffadca30474f977acbe86ce9f682 (DEFAULT_TABLE table=test-table [id=b11d2e0963e242889782bbe6048bff5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:55.066239 29886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5831ffadca30474f977acbe86ce9f682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:55.066748 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.059410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59114 (local address 127.10.207.3:33973)
0807 08:39:55.059769 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:55.059776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:55.059791 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:39:55.060211 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:55.060216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:55.060278 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:55.060353 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:55.060611 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.061708 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.063761 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.064510 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.064587 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.064642 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:55.066616 (+ 1974us) server_negotiation.cc:299] Negotiation successful
0807 08:39:55.066659 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":70,"threads_started":1}
I20260807 08:39:55.070057 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.059199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:44543 (local address 127.0.0.1:38434)
0807 08:39:55.063653 (+ 4454us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:55.063662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:55.063700 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:55.064953 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:55.064954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:55.064957 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:55.065002 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.065004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.067446 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.067447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.068869 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.068871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.069837 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.069838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.069899 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.069903 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:39:55.069950 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:39:55.069985 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4438,"thread_start_us":80,"threads_started":1}
I20260807 08:39:55.070326 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.059421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.10.207.2:44543)
0807 08:39:55.064709 (+ 5288us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:55.064714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:55.064724 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:39:55.064759 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:55.064761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:55.064777 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:55.064863 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:55.065751 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.067354 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.068951 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.069673 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.069740 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.069787 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:55.070216 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 08:39:55.070256 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5257,"thread_start_us":74,"threads_started":1}
I20260807 08:39:55.070976 30101 tablet_service.cc:1467] Processing CreateTablet for tablet 5831ffadca30474f977acbe86ce9f682 (DEFAULT_TABLE table=test-table [id=b11d2e0963e242889782bbe6048bff5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:55.071009 30305 tablet_service.cc:1467] Processing CreateTablet for tablet 5831ffadca30474f977acbe86ce9f682 (DEFAULT_TABLE table=test-table [id=b11d2e0963e242889782bbe6048bff5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:39:55.071367 30305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5831ffadca30474f977acbe86ce9f682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:55.071373 30101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5831ffadca30474f977acbe86ce9f682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:55.074702 30463 tablet_bootstrap.cc:492] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Bootstrap starting.
I20260807 08:39:55.074713 30462 tablet_bootstrap.cc:492] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf: Bootstrap starting.
I20260807 08:39:55.075143 30465 tablet_bootstrap.cc:492] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Bootstrap starting.
I20260807 08:39:55.075479 30462 tablet_bootstrap.cc:654] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:55.075723 30465 tablet_bootstrap.cc:654] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:55.075810 30462 log.cc:824] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:55.076108 30465 log.cc:824] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:55.077092 30465 tablet_bootstrap.cc:492] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: No bootstrap required, opened a new log
I20260807 08:39:55.077165 30465 ts_tablet_manager.cc:1397] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:39:55.079100 30462 tablet_bootstrap.cc:492] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf: No bootstrap required, opened a new log
I20260807 08:39:55.079188 30462 ts_tablet_manager.cc:1397] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 08:39:55.079850 30465 raft_consensus.cc:348] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.080034 30465 raft_consensus.cc:374] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:55.080080 30465 raft_consensus.cc:729] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc06c11486b409785a372c55295aa43, State: Initialized, Role: FOLLOWER
I20260807 08:39:55.079144 30463 tablet_bootstrap.cc:654] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:55.080229 30465 consensus_queue.cc:260] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.080538 30465 ts_tablet_manager.cc:1428] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 08:39:55.080667 30197 heartbeater.cc:503] Master 127.10.207.62:34327 was elected leader, sending a full tablet report...
I20260807 08:39:55.081043 30463 log.cc:824] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:55.081602 30462 raft_consensus.cc:348] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.081786 30462 raft_consensus.cc:374] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:55.081836 30462 raft_consensus.cc:729] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5475b36663574de6bb7d586c8bf3accf, State: Initialized, Role: FOLLOWER
I20260807 08:39:55.082679 30462 consensus_queue.cc:260] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.082966 30462 ts_tablet_manager.cc:1428] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:39:55.083120 29996 heartbeater.cc:503] Master 127.10.207.62:34327 was elected leader, sending a full tablet report...
I20260807 08:39:55.083345 30463 tablet_bootstrap.cc:492] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: No bootstrap required, opened a new log
I20260807 08:39:55.083426 30463 ts_tablet_manager.cc:1397] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260807 08:39:55.087124 30463 raft_consensus.cc:348] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.087332 30463 raft_consensus.cc:374] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:55.087383 30463 raft_consensus.cc:729] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c883ab31e27e4fd6a11aa245f0ee146b, State: Initialized, Role: FOLLOWER
I20260807 08:39:55.087543 30463 consensus_queue.cc:260] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.087837 30463 ts_tablet_manager.cc:1428] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:39:55.087966 30411 heartbeater.cc:503] Master 127.10.207.62:34327 was elected leader, sending a full tablet report...
W20260807 08:39:55.127000 29997 tablet.cc:2369] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:39:55.248236 30412 tablet.cc:2369] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:39:55.284309 30472 raft_consensus.cc:482] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:39:55.284508 30472 raft_consensus.cc:504] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.285040 30472 leader_election.cc:288] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5475b36663574de6bb7d586c8bf3accf (127.10.207.1:41903), adc06c11486b409785a372c55295aa43 (127.10.207.2:44543)
I20260807 08:39:55.290222 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.286302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:34515 (local address 127.10.207.2:44543)
0807 08:39:55.286373 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:55.286376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:55.286455 (+ 79us) server_negotiation.cc:407] Connection header received
0807 08:39:55.286601 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:55.286602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:55.286614 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:55.286646 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:55.286900 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.287710 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.289186 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.289759 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.289814 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.289830 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:55.290087 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 08:39:55.290140 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:39:55.290794 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.286057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:44543 (local address 127.10.207.3:34515)
0807 08:39:55.286379 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:55.286390 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:55.286485 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:55.286725 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:55.286726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:55.286728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:55.286822 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.286824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.287811 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.287812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.289074 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.289078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.289919 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.289920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.289991 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.289995 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:39:55.290621 (+ 626us) client_negotiation.cc:240] Negotiation successful
0807 08:39:55.290678 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":102,"threads_started":1}
I20260807 08:39:55.291378 30130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831ffadca30474f977acbe86ce9f682" candidate_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc06c11486b409785a372c55295aa43" is_pre_election: true
I20260807 08:39:55.291628 30130 raft_consensus.cc:2454] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c883ab31e27e4fd6a11aa245f0ee146b in term 0.
I20260807 08:39:55.292034 30234 leader_election.cc:302] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adc06c11486b409785a372c55295aa43, c883ab31e27e4fd6a11aa245f0ee146b; no voters:
I20260807 08:39:55.292229 30472 raft_consensus.cc:2781] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:39:55.292300 30472 raft_consensus.cc:482] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:39:55.292340 30472 raft_consensus.cc:3037] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:55.293406 30472 raft_consensus.cc:504] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.293586 30472 leader_election.cc:288] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [CANDIDATE]: Term 1 election: Requested vote from peers 5475b36663574de6bb7d586c8bf3accf (127.10.207.1:41903), adc06c11486b409785a372c55295aa43 (127.10.207.2:44543)
I20260807 08:39:55.293855 30130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831ffadca30474f977acbe86ce9f682" candidate_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc06c11486b409785a372c55295aa43"
I20260807 08:39:55.293946 30130 raft_consensus.cc:3037] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:55.294140 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.285560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:38447 (local address 127.10.207.1:41903)
0807 08:39:55.285617 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:55.285620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:55.285634 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:39:55.285681 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:55.285682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:55.285695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:55.285778 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:39:55.289265 (+ 3487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.290028 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.293072 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.293693 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.293750 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.293761 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:55.294006 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 08:39:55.294066 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:39:55.294387 30475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.285291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41903 (local address 127.10.207.3:38447)
0807 08:39:55.285530 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:55.285544 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:55.285621 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:55.286284 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:55.286286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:55.286290 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:39:55.289130 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.289136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.290149 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.290150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.292936 (+ 2786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:55.292940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.293830 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.293831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:55.293913 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.293918 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:39:55.294244 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 08:39:55.294305 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":151,"threads_started":1}
I20260807 08:39:55.294864 30130 raft_consensus.cc:2454] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c883ab31e27e4fd6a11aa245f0ee146b in term 1.
I20260807 08:39:55.295009 29922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831ffadca30474f977acbe86ce9f682" candidate_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5475b36663574de6bb7d586c8bf3accf" is_pre_election: true
I20260807 08:39:55.295070 30234 leader_election.cc:302] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adc06c11486b409785a372c55295aa43, c883ab31e27e4fd6a11aa245f0ee146b; no voters:
I20260807 08:39:55.295168 30472 raft_consensus.cc:2781] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:55.295202 29922 raft_consensus.cc:2454] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c883ab31e27e4fd6a11aa245f0ee146b in term 0.
I20260807 08:39:55.295377 30472 raft_consensus.cc:686] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 1 LEADER]: Becoming Leader. State: Replica: c883ab31e27e4fd6a11aa245f0ee146b, State: Running, Role: LEADER
I20260807 08:39:55.295360 29921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831ffadca30474f977acbe86ce9f682" candidate_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5475b36663574de6bb7d586c8bf3accf"
I20260807 08:39:55.295439 29921 raft_consensus.cc:3037] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:55.295497 30472 consensus_queue.cc:237] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } }
I20260807 08:39:55.296517 29921 raft_consensus.cc:2454] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c883ab31e27e4fd6a11aa245f0ee146b in term 1.
I20260807 08:39:55.296509 29675 catalog_manager.cc:5495] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b reported cstate change: term changed from 0 to 1, leader changed from <none> to c883ab31e27e4fd6a11aa245f0ee146b (127.10.207.3). New cstate: current_term: 1 leader_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c883ab31e27e4fd6a11aa245f0ee146b" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 } health_report { overall_health: UNKNOWN } } }
W20260807 08:39:55.315514 30199 tablet.cc:2369] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 08:39:55 dist-test-slave-h6n0 krb5kdc[28095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091994, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for HTTP/127.10.207.20@KRBTEST.COM
I20260807 08:39:55.349716 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.346135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.10.207.3:33973)
0807 08:39:55.346189 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:55.346191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:55.346203 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:39:55.346255 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:55.346256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:55.346283 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:55.346381 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:39:55.346630 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.347355 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.348424 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:55.348716 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:55.348785 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:55.349014 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:39:55.349178 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:39:55.349282 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:55.349565 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 08:39:55.349593 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:39:55.352737 30130 raft_consensus.cc:1261] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 1 FOLLOWER]: Refusing update from remote peer c883ab31e27e4fd6a11aa245f0ee146b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:55.353171 29921 raft_consensus.cc:1261] T 5831ffadca30474f977acbe86ce9f682 P 5475b36663574de6bb7d586c8bf3accf [term 1 FOLLOWER]: Refusing update from remote peer c883ab31e27e4fd6a11aa245f0ee146b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:55.353868 30472 consensus_queue.cc:1035] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [LEADER]: Connected to new peer: Peer: permanent_uuid: "adc06c11486b409785a372c55295aa43" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 44543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:55.354040 30478 consensus_queue.cc:1035] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5475b36663574de6bb7d586c8bf3accf" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:55.358922 30482 mvcc.cc:205] Tried to move back new op lower bound from 7315832812960759808 to 7315832812730945536. Current Snapshot: MvccSnapshot[applied={T|T < 7315832812960759808}]
I20260807 08:39:55.359849 30483 mvcc.cc:205] Tried to move back new op lower bound from 7315832812960759808 to 7315832812730945536. Current Snapshot: MvccSnapshot[applied={T|T < 7315832812960759808}]
I20260807 08:39:55.360822 30489 mvcc.cc:205] Tried to move back new op lower bound from 7315832812960759808 to 7315832812730945536. Current Snapshot: MvccSnapshot[applied={T|T < 7315832812960759808}]
I20260807 08:39:55.364116 29674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:55.364244 29674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:39:55.365103 29674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1af7da3c5374088b826cc3497c12b27: Sending DeleteTablet for 3 replicas of tablet 5831ffadca30474f977acbe86ce9f682
I20260807 08:39:55.365547 29886 tablet_service.cc:1514] Processing DeleteTablet for tablet 5831ffadca30474f977acbe86ce9f682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:55 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:45968
I20260807 08:39:55.366290 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 29743
I20260807 08:39:55.372402 30305 tablet_service.cc:1514] Processing DeleteTablet for tablet 5831ffadca30474f977acbe86ce9f682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:55 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:59114
I20260807 08:39:55.372867 30101 tablet_service.cc:1514] Processing DeleteTablet for tablet 5831ffadca30474f977acbe86ce9f682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:39:55 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:38434
I20260807 08:39:55.373211 30504 tablet_replica.cc:330] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: stopping tablet replica
I20260807 08:39:55.373377 30504 raft_consensus.cc:2229] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:39:55.373592 30504 raft_consensus.cc:2258] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:55.374234 30504 ts_tablet_manager.cc:1905] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:55.374610 30502 tablet_replica.cc:330] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: stopping tablet replica
I20260807 08:39:55.374799 30502 raft_consensus.cc:2229] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:39:55.375265 30502 raft_consensus.cc:2258] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:39:55.377812 30504 ts_tablet_manager.cc:1918] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:55.377967 30504 log.cc:1196] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/5831ffadca30474f977acbe86ce9f682
I20260807 08:39:55.378407 30504 ts_tablet_manager.cc:1939] T 5831ffadca30474f977acbe86ce9f682 P adc06c11486b409785a372c55295aa43: Deleting consensus metadata
I20260807 08:39:55.379077 29651 catalog_manager.cc:4841] TS adc06c11486b409785a372c55295aa43 (127.10.207.2:44543): tablet 5831ffadca30474f977acbe86ce9f682 (table test-table [id=b11d2e0963e242889782bbe6048bff5b]) successfully deleted
I20260807 08:39:55.379542 30502 ts_tablet_manager.cc:1905] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:39:55.381806 30502 ts_tablet_manager.cc:1918] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:39:55.381948 30502 log.cc:1196] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/5831ffadca30474f977acbe86ce9f682
I20260807 08:39:55.382290 30502 ts_tablet_manager.cc:1939] T 5831ffadca30474f977acbe86ce9f682 P c883ab31e27e4fd6a11aa245f0ee146b: Deleting consensus metadata
I20260807 08:39:55.382804 29651 catalog_manager.cc:4841] TS c883ab31e27e4fd6a11aa245f0ee146b (127.10.207.3:33973): tablet 5831ffadca30474f977acbe86ce9f682 (table test-table [id=b11d2e0963e242889782bbe6048bff5b]) successfully deleted
W20260807 08:39:55.385814 29649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:41903 (error 108)
I20260807 08:39:55.386657 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:55.386473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41903 (local address 127.0.0.1:45980)
0807 08:39:55.386557 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:55.386606 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:41903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:39:55.386704 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 29999
W20260807 08:39:55.392702 29649 catalog_manager.cc:4584] TS 5475b36663574de6bb7d586c8bf3accf (127.10.207.1:41903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5831ffadca30474f977acbe86ce9f682: Network error: Client connection negotiation failed: client connection to 127.10.207.1:41903: connect: Connection refused (error 111)
I20260807 08:39:55.398488 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 30204
I20260807 08:39:55.411062 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 29626
I20260807 08:39:55.428980 11068 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:39:55.777968 11068 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:39:55.778098 11068 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:39:56.123518 11068 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:39:56.123 UTC [28146] LOG: received smart shutdown request
2026-08-07 08:39:56.128 UTC [28146] LOG: background worker "logical replication launcher" (PID 28162) exited with exit code 1
2026-08-07 08:39:56.129 UTC [28151] LOG: shutting down
2026-08-07 08:39:56.288 UTC [28146] LOG: database system is shut down
2026-08-07T08:39:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52070 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:56 dist-test-slave-h6n0 krb5kdc[30525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:56 dist-test-slave-h6n0 krb5kdc[30525](info): set up 2 sockets
Aug 07 08:39:56 dist-test-slave-h6n0 krb5kdc[30525](info): commencing operation
krb5kdc: starting...
W20260807 08:39:57.682618 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:39:57 dist-test-slave-h6n0 krb5kdc[30525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:39:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:39:57.816944 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:38231
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:32845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:38231,127.10.207.61:41855,127.10.207.60:38339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:39:57.910738 30543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:57.911016 30543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:57.911054 30543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:57.912261 30543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:57.912329 30543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:57.912357 30543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:57.912390 30543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:57.912423 30543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:57.913945 30543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:32845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:38231,127.10.207.61:41855,127.10.207.60:38339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:38231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:57.914251 30543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:57.914502 30543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:57.919852 30554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:57.919893 30559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:57.919898 30555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:57.920260 30543 server_base.cc:1034] running on GCE node
I20260807 08:39:57.920527 30543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:57.920799 30543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:57.921936 30543 hybrid_clock.cc:648] HybridClock initialized: now 1786091997921917 us; error 42 us; skew 500 ppm
Aug 07 08:39:57 dist-test-slave-h6n0 krb5kdc[30525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:57.924790 30543 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:39:57.925339 30543 webserver.cc:460] Webserver started at http://127.10.207.62:37865/ using document root <none> and password file <none>
I20260807 08:39:57.925585 30543 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:57.925654 30543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:57.925853 30543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:57.927114 30543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "0d410d3b1fd34d65af61e38a68e0c464"
format_stamp: "Formatted at 2026-08-07 08:39:57 on dist-test-slave-h6n0"
server_key: "38d787b5825f7ead2443a860df6bf374"
server_key_iv: "019130f0509910de4d1c6c8609d93603"
server_key_version: "encryptionkey@0"
I20260807 08:39:57.927520 30543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "0d410d3b1fd34d65af61e38a68e0c464"
format_stamp: "Formatted at 2026-08-07 08:39:57 on dist-test-slave-h6n0"
server_key: "38d787b5825f7ead2443a860df6bf374"
server_key_iv: "019130f0509910de4d1c6c8609d93603"
server_key_version: "encryptionkey@0"
I20260807 08:39:57.929059 30543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:39:57.929903 30572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:57.930131 30543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:39:57.930251 30543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "0d410d3b1fd34d65af61e38a68e0c464"
format_stamp: "Formatted at 2026-08-07 08:39:57 on dist-test-slave-h6n0"
server_key: "38d787b5825f7ead2443a860df6bf374"
server_key_iv: "019130f0509910de4d1c6c8609d93603"
server_key_version: "encryptionkey@0"
I20260807 08:39:57.930330 30543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:57.946264 30543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:57.948454 30543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:57.948669 30543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:57.954161 30543 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:38231
I20260807 08:39:57.954247 30643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:38231 every 8 connection(s)
I20260807 08:39:57.954665 30543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:39:57.955772 30645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:57.957485 30645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:57.963027 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 30543
I20260807 08:39:57.963131 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:39:57.963340 11068 external_mini_cluster.cc:1428] Setting key 12fdad9fa87554870e69824af541d95e
Aug 07 08:39:57 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:57.968365 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:57.958288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.10.207.62:38231)
0807 08:39:57.958475 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:57.958481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:57.958620 (+ 139us) server_negotiation.cc:407] Connection header received
0807 08:39:57.959264 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:57.959266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:57.959298 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:57.959391 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:57.959990 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:57.960834 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:57.961775 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:57.962106 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:57.962189 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:57.964456 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:57.964464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:57.964465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:57.964474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:57.966354 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:57.966925 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:57.966927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:57.966927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:57.967028 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:57.967358 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:57.967359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:57.967359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:57.967576 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:57.967644 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:57.967908 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 08:39:57.968165 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260807 08:39:57.968420 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:57.958237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38231 (local address 127.0.0.1:57214)
0807 08:39:57.958422 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:57.958468 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:57.959098 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:57.959429 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:57.959432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:57.959697 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:57.959896 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:57.959900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:57.960929 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:57.960930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:57.961695 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:57.961697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:57.962189 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:57.962190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:57.962259 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:57.962767 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:57.962776 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:57.964357 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:57.966445 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:57.966453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:57.966455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:57.966789 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:57.967120 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:57.967121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:57.967122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:57.967274 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:57.967651 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:57.967658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:57.967742 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 08:39:57.967840 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 08:39:57.968091 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":5,"thread_start_us":98,"threads_started":1}
I20260807 08:39:57.969571 30645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:57.969969 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:57.969808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54746)
0807 08:39:57.969871 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:57.969934 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 08:39:57.970168 30574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
I20260807 08:39:57.970713 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:57.970604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54754)
0807 08:39:57.970648 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:57.970683 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:39:57.970929 30645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:41855: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.207.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab.
Entry for principal kudu/127.10.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab.
I20260807 08:39:57.993407 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:41855
--webserver_interface=127.10.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab
--principal=kudu/127.10.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:32845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:38231,127.10.207.61:41855,127.10.207.60:38339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:39:58.020170 30645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } attempt: 1
I20260807 08:39:58.020797 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.020641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54764)
0807 08:39:58.020719 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.020767 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 08:39:58.021229 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.021133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54780)
0807 08:39:58.021185 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.021213 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:39:58.021355 30645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:41855: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
W20260807 08:39:58.088447 30656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:58.088675 30656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:58.088699 30656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 08:39:58.089515 30645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } attempt: 2
I20260807 08:39:58.090020 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.089883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54788)
0807 08:39:58.089954 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.089992 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:39:58.090077 30656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:58.090138 30656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:58.090178 30656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:58.090196 30656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:58.090216 30656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:58.090425 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.090320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54804)
0807 08:39:58.090374 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.090399 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 08:39:58.090615 30645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:41855: Network error: Client connection negotiation failed: client connection to 127.10.207.61:41855: connect: Connection refused (error 111)
I20260807 08:39:58.091948 30656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:32845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:38231,127.10.207.61:41855,127.10.207.60:38339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab
--principal=kudu/127.10.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:41855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:58.092240 30656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:58.092502 30656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:58.097893 30663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:58.097927 30665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:58.097935 30662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:58.098340 30656 server_base.cc:1034] running on GCE node
I20260807 08:39:58.098673 30656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:58.099057 30656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:58.100220 30656 hybrid_clock.cc:648] HybridClock initialized: now 1786091998100214 us; error 36 us; skew 500 ppm
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:58.103281 30656 init.cc:377] Logged in from keytab as kudu/127.10.207.61@KRBTEST.COM (short username kudu)
I20260807 08:39:58.103762 30656 webserver.cc:460] Webserver started at http://127.10.207.61:43409/ using document root <none> and password file <none>
I20260807 08:39:58.104040 30656 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:58.104112 30656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:58.104270 30656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:58.105389 30656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data/instance:
uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4"
format_stamp: "Formatted at 2026-08-07 08:39:58 on dist-test-slave-h6n0"
server_key: "257d6bddaca61fc00e40925b0ab030c3"
server_key_iv: "5d00b797b2727cbc52e47859cd62290f"
server_key_version: "encryptionkey@0"
I20260807 08:39:58.105779 30656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/wal/instance:
uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4"
format_stamp: "Formatted at 2026-08-07 08:39:58 on dist-test-slave-h6n0"
server_key: "257d6bddaca61fc00e40925b0ab030c3"
server_key_iv: "5d00b797b2727cbc52e47859cd62290f"
server_key_version: "encryptionkey@0"
I20260807 08:39:58.107379 30656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:39:58.108250 30678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:58.108500 30656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:39:58.108625 30656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/wal
uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4"
format_stamp: "Formatted at 2026-08-07 08:39:58 on dist-test-slave-h6n0"
server_key: "257d6bddaca61fc00e40925b0ab030c3"
server_key_iv: "5d00b797b2727cbc52e47859cd62290f"
server_key_version: "encryptionkey@0"
I20260807 08:39:58.108709 30656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:58.129037 30656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:58.131372 30656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:58.131541 30656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:58.136170 30656 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.61:41855
I20260807 08:39:58.136291 30769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.61:41855 every 8 connection(s)
I20260807 08:39:58.136611 30656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
I20260807 08:39:58.137665 30770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:58.139986 30770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:58.140641 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 30656
I20260807 08:39:58.140772 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-1/wal/instance
I20260807 08:39:58.140986 11068 external_mini_cluster.cc:1428] Setting key 0f5741f7868c35ea246ab871209a1ae9
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.207.60@KRBTEST.COM; defaulting to no policy
I20260807 08:39:58.150578 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.140872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57226 (local address 127.10.207.62:38231)
0807 08:39:58.140942 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.140945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.141195 (+ 250us) server_negotiation.cc:407] Connection header received
0807 08:39:58.141774 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.141775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.141793 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.141846 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.142601 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.143159 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.144473 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.144721 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.144796 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.147588 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.147589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.147589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.147598 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.149087 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.149544 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.149546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.149546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.149609 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.149875 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.149876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.149876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.149998 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.150077 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.150363 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.150448 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:39:58.150800 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.140833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38231 (local address 127.0.0.1:57226)
0807 08:39:58.141034 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.141082 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.141656 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.141945 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.141949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.142243 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.142469 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.142473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.143265 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.143266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.144356 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.144359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.144831 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.144832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.145007 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.145529 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.145541 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.147445 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.149200 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.149208 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.149211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.149461 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.149682 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.149683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.149683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.149799 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.150069 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.150077 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.150220 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.150325 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.150522 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260807 08:39:58.151517 30770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.60@KRBTEST.COM" created.
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:39:58.159813 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.151829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54814)
0807 08:39:58.151887 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.151898 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.151941 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.152280 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.152281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.152439 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.152493 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.152494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.153472 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.153473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.153984 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.153985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.154496 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.154497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.154598 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.155075 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.155081 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.156525 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.158350 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.158352 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.158352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.158635 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.158873 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.158874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.158875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.158949 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.159273 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.159274 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.159332 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.159424 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.159570 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:39:58.159832 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.151932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54814 (local address 127.10.207.61:41855)
0807 08:39:58.152133 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.152137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.152155 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:39:58.152173 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.152175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.152186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.152280 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.152587 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.153376 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.154090 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.154405 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.154497 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.156640 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.156647 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.156647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.156654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.158256 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.158731 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.158734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.158734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.158792 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.159029 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.159029 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.159030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.159187 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.159258 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.159461 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.159640 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260807 08:39:58.160656 30770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:58.161077 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.160920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49608)
0807 08:39:58.160984 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.161044 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 08:39:58.161249 30686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
I20260807 08:39:58.161784 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.161653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49610)
0807 08:39:58.161716 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.161751 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 08:39:58.161927 30770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38339: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab.
Entry for principal kudu/127.10.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab.
I20260807 08:39:58.169677 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:38339
--webserver_interface=127.10.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab
--principal=kudu/127.10.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:32845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:38231,127.10.207.61:41855,127.10.207.60:38339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:39:58.181752 30645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } attempt: 3
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:39:58.188752 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.182161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.10.207.61:41855)
0807 08:39:58.182245 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.182246 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.182253 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:39:58.182271 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.182272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.182284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.182348 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.182700 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.183223 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.183992 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.184210 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.184287 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.186132 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.186133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.186134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.186141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.187565 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.187943 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.187944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.187944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.188002 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.188238 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.188239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.188239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.188348 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.188427 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.188587 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.188663 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:39:58.188753 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.182044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54830)
0807 08:39:58.182101 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.182118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.182155 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.182353 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.182354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.182499 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.182613 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.182615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.183309 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.183310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.183910 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.183912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.184286 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.184287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.184370 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.184897 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.184901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.186041 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.187695 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.187696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.187696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.187868 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.188106 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.188107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.188107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.188171 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.188420 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.188421 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.188467 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.188561 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.188661 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:39:58.189420 30645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:39:58.189792 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.189691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49616)
0807 08:39:58.189741 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.189768 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:39:58.190094 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.190007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49618)
0807 08:39:58.190053 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.190075 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 08:39:58.190194 30645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38339: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
I20260807 08:39:58.211148 30770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } attempt: 1
I20260807 08:39:58.211798 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.211632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49634)
0807 08:39:58.211709 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.211762 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 08:39:58.212268 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.212160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49638)
0807 08:39:58.212217 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.212251 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 08:39:58.212392 30770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38339: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
I20260807 08:39:58.221395 30645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } attempt: 1
I20260807 08:39:58.221974 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.221826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49648)
0807 08:39:58.221897 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.221943 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 08:39:58.222572 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.222385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49650)
0807 08:39:58.222453 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.222486 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:39:58.222709 30645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38339: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
W20260807 08:39:58.266019 30780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:39:58.266292 30780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:39:58.266319 30780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:39:58.267627 30780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:39:58.267674 30780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:39:58.267692 30780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:39:58.267705 30780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:39:58.267719 30780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:39:58.269431 30780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:32845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:38231,127.10.207.61:41855,127.10.207.60:38339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab
--principal=kudu/127.10.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:38339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:39:58.269743 30780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:39:58.270042 30780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:39:58.277854 30794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:39:58.278088 30788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:58.278475 30780 server_base.cc:1034] running on GCE node
W20260807 08:39:58.278715 30789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:39:58.279325 30780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:39:58.279853 30780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:39:58.280565 30770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } attempt: 2
I20260807 08:39:58.281025 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.280904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49664)
0807 08:39:58.280949 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.280981 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 08:39:58.281064 30780 hybrid_clock.cc:648] HybridClock initialized: now 1786091998281021 us; error 74 us; skew 500 ppm
I20260807 08:39:58.281412 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.281314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49674)
0807 08:39:58.281367 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.281391 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:39:58.281541 30770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38339: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:39:58.284632 30780 init.cc:377] Logged in from keytab as kudu/127.10.207.60@KRBTEST.COM (short username kudu)
I20260807 08:39:58.285293 30780 webserver.cc:460] Webserver started at http://127.10.207.60:36961/ using document root <none> and password file <none>
I20260807 08:39:58.285584 30780 fs_manager.cc:359] Metadata directory not provided
I20260807 08:39:58.285678 30780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:39:58.285838 30780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:39:58.287185 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data/instance:
uuid: "d57a1f63dc974e11aac9ec2f0d0053c7"
format_stamp: "Formatted at 2026-08-07 08:39:58 on dist-test-slave-h6n0"
server_key: "fa222d1a1114a5bed2c48b56b3fc63ee"
server_key_iv: "0418f9332cd851ee7bf06c9ac2e4a26c"
server_key_version: "encryptionkey@0"
I20260807 08:39:58.287599 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/wal/instance:
uuid: "d57a1f63dc974e11aac9ec2f0d0053c7"
format_stamp: "Formatted at 2026-08-07 08:39:58 on dist-test-slave-h6n0"
server_key: "fa222d1a1114a5bed2c48b56b3fc63ee"
server_key_iv: "0418f9332cd851ee7bf06c9ac2e4a26c"
server_key_version: "encryptionkey@0"
I20260807 08:39:58.289330 30780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:39:58.290422 30803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:39:58.290726 30780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:39:58.290832 30780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/wal
uuid: "d57a1f63dc974e11aac9ec2f0d0053c7"
format_stamp: "Formatted at 2026-08-07 08:39:58 on dist-test-slave-h6n0"
server_key: "fa222d1a1114a5bed2c48b56b3fc63ee"
server_key_iv: "0418f9332cd851ee7bf06c9ac2e4a26c"
server_key_version: "encryptionkey@0"
I20260807 08:39:58.290934 30780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:39:58.297871 30645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } attempt: 2
I20260807 08:39:58.298678 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.298429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49682)
0807 08:39:58.298516 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.298626 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 08:39:58.299101 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.299001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49688)
0807 08:39:58.299054 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.299083 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:39:58.299300 30645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38339: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38339: connect: Connection refused (error 111)
I20260807 08:39:58.307147 30780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:39:58.309635 30780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:39:58.309813 30780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:39:58.315337 30780 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.60:38339
I20260807 08:39:58.315380 30881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.60:38339 every 8 connection(s)
I20260807 08:39:58.315865 30780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
I20260807 08:39:58.317005 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:39:58.318820 30882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:39:58.325968 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 30780
I20260807 08:39:58.326107 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091830136701-11068-0/minicluster-data/master-2/wal/instance
I20260807 08:39:58.326403 11068 external_mini_cluster.cc:1428] Setting key d00807303b3e8f94f8eea17c99d649c4
I20260807 08:39:58.329263 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.319731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57240 (local address 127.10.207.62:38231)
0807 08:39:58.319858 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.319861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.320131 (+ 270us) server_negotiation.cc:407] Connection header received
0807 08:39:58.320812 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.320814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.320834 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.320915 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.321595 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.322092 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.323180 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.323365 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.323472 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.325799 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.325802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.325802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.325812 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.327812 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.328254 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.328255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.328256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.328331 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.328604 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.328605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.328605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.328722 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.328794 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.329093 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.329171 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 08:39:58.329468 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.319631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38231 (local address 127.0.0.1:57240)
0807 08:39:58.319905 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.319969 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.320592 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.320991 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.320996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.321295 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.321493 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.321497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.322202 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.322203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.323075 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.323077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.323482 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.323483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.323618 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.324119 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.324128 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.325662 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.327897 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.327904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.327906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.328154 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.328408 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.328409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.328409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.328526 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.328798 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.328805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.328939 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.329067 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.329229 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":108,"threads_started":1}
I20260807 08:39:58.330034 30882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:39:58.336475 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.326978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.10.207.60:38339)
0807 08:39:58.327126 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.327130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.327332 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:39:58.327362 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.327364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.327378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.327479 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.328173 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.328938 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.330007 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.330511 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.330627 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.333143 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.333151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.333152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.333159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.334871 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.335335 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.335339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.335340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.335448 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.335769 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.335770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.335770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.335938 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.336073 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.336280 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.336368 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260807 08:39:58.337776 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.330299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:41855 (local address 127.0.0.1:54834)
0807 08:39:58.330349 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.330359 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.330395 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.330624 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.330625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.330761 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.330815 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.330816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.331430 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.331431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.331922 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.331923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.332309 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.332310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.332385 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.332825 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.332830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.334464 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.336319 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.336320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.336321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.336784 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.336998 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.336998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.336999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.337087 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.337431 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.337432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.337480 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.337574 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.337687 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:39:58.337800 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.330403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.10.207.61:41855)
0807 08:39:58.330488 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.330491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.330499 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:39:58.330556 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.330557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.330568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.330621 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.330894 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.331350 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.332007 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.332220 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.332307 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.334614 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.334619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.334623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.334632 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.336230 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.336849 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.336851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.336851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.336911 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.337180 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.337182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.337182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.337346 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.337439 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.337603 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.337703 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:39:58.338470 30882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:39:58.345857 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.337445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57256 (local address 127.10.207.62:38231)
0807 08:39:58.337484 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.337486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.337494 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:39:58.337563 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.337564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.337573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.337616 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.338333 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.339095 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.339817 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.340076 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.340166 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.342828 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.342829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.342829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.342836 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.344597 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.344968 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.344969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.344969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.345033 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.345259 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.345261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.345261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.345373 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.345571 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.345609 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.345690 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:39:58.346166 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.338734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49708)
0807 08:39:58.338793 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.338803 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.338839 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.338984 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.338984 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.339099 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.339153 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.339155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.339926 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.339927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.340571 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.340574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.340946 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.340947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.341032 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.341376 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.341381 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.343191 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.345080 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.345081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.345082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.345271 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.345511 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.345512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.345513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.345570 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.345836 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.345838 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.345884 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.345956 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.346079 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 08:39:58.346381 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.338814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.10.207.60:38339)
0807 08:39:58.338853 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.338854 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.338865 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:39:58.338891 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.338892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.338904 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.338977 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.339272 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.339829 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.340662 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.340859 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.340917 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.343339 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.343341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.343341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.343349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.344979 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.345378 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.345379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.345380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.345444 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.345641 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.345642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.345643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.345762 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.345834 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.345982 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.346301 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":226,"server-negotiator.queue_time_us":20}
I20260807 08:39:58.348392 30882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: Bootstrap starting.
I20260807 08:39:58.349781 30882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:58.350147 30882 log.cc:824] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:58.351204 30882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: No bootstrap required, opened a new log
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:39:58.352739 30882 raft_consensus.cc:348] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.352859 30882 raft_consensus.cc:374] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:58.352883 30882 raft_consensus.cc:729] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d57a1f63dc974e11aac9ec2f0d0053c7, State: Initialized, Role: FOLLOWER
I20260807 08:39:58.353016 30882 consensus_queue.cc:260] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.353368 30882 sys_catalog.cc:564] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:58.353382 30894 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.353562 30894 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:39:58.355325 30914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 08:39:58.355378 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.346799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54850 (local address 127.10.207.61:41855)
0807 08:39:58.346881 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.346884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.346893 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:39:58.346916 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.346917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.346928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.346991 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.347589 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.348031 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.349330 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.349528 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.349601 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.352284 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.352285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.352286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.352293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.354070 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.354485 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.354487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.354487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.354596 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.354814 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.354815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.354815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.354964 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.355156 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.355195 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.355271 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 08:39:58.355391 30914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:58.357122 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:39:58.355901 30780 master_runner.cc:186] Error getting master registration for 127.10.207.62:38231: OK, instance_id { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" instance_seqno: 1786091997952647 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:39:58.359337 30780 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:39:58.368237 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.358995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.10.207.60:38339)
0807 08:39:58.359036 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.359052 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.359554 (+ 502us) server_negotiation.cc:407] Connection header received
0807 08:39:58.359659 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.359660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.359672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.359703 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.360564 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.361321 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.362367 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.362688 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.362730 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.364748 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.364749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.364750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.364757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.366729 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.367157 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.367158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.367158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.367211 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.367455 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.367455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.367456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.367644 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.367738 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.368000 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.368120 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:39:58.368242 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.358885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.10.207.62:38231)
0807 08:39:58.359018 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.359020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.359334 (+ 314us) server_negotiation.cc:407] Connection header received
0807 08:39:58.359468 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.359469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.359482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.359556 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.360371 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.361265 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.362489 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.362895 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.362958 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.364826 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.364827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.364827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.364835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.366920 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.367347 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.367348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.367348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.367407 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.367623 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.367624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.367624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.367744 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.367817 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.368064 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.368144 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 08:39:58.369194 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.358938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.10.207.61:41855)
0807 08:39:58.358981 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.358983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.359151 (+ 168us) server_negotiation.cc:407] Connection header received
0807 08:39:58.359235 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.359236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.359248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.359308 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.360493 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.361280 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.362420 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.362751 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.362790 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.364769 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.364770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.364771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.364777 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.366639 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.367147 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.367148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.367148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.367211 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.367465 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.367465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.367466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.367586 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.368222 (+ 636us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.368899 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.368981 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:39:58.372684 30770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } attempt: 3
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091998, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:39:58.380401 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.373011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49728)
0807 08:39:58.373093 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.373104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.373150 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.373399 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.373401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.373599 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.373651 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.373652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.374412 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.374414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.375270 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.375272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.375619 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.375620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.375761 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.376326 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.376332 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.377684 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.379345 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.379346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.379347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.379526 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.379745 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.379746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.379746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.379811 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.380058 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.380059 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.380105 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.380183 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.380290 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 08:39:58.380440 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.373075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49728 (local address 127.10.207.60:38339)
0807 08:39:58.373143 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.373145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.373158 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:39:58.373258 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.373260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.373275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.373348 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.373720 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.374321 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.375365 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.375542 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.375606 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.377773 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.377775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.377775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.377782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.379243 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.379620 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.379622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.379622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.379675 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.379879 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.379880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.379880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.379986 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.380053 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.380240 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.380350 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:39:58.381923 30770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: Bootstrap starting.
I20260807 08:39:58.382762 30770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:58.383124 30770 log.cc:824] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:58.383937 30770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: No bootstrap required, opened a new log
I20260807 08:39:58.385984 30770 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.386148 30770 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:58.386191 30770 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b91f43c4dc409fb4c6bebd3f50bcd4, State: Initialized, Role: FOLLOWER
I20260807 08:39:58.386363 30770 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.388832 30770 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:58.388825 30931 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.389050 30931 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:39:58.391408 30946 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:58.391532 30946 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:58.391539 30656 master_runner.cc:186] Error getting master registration for 127.10.207.62:38231: OK, instance_id { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" instance_seqno: 1786091997952647 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:39:58.393447 30656 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:39:58.398496 30645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } attempt: 3
Aug 07 08:39:58 dist-test-slave-h6n0 krb5kdc[30525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091997, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:39:58.405839 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.398925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38339 (local address 127.0.0.1:49734)
0807 08:39:58.398992 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 08:39:58.399002 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:39:58.399038 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:39:58.399302 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:39:58.399303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:39:58.399487 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:39:58.399540 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.399541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.400233 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.400234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.400752 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:39:58.400753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.401119 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.401120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:39:58.401181 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.401686 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:39:58.401691 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:39:58.403046 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:39:58.404691 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.404692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.404693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.404927 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.405138 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:39:58.405139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:39:58.405140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:39:58.405205 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:39:58.405470 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:39:58.405471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:39:58.405520 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 08:39:58.405623 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:39:58.405735 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 08:39:58.405839 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:39:58.399121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.10.207.60:38339)
0807 08:39:58.399183 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:39:58.399185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:39:58.399199 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:39:58.399221 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:39:58.399222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:39:58.399230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:39:58.399295 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:39:58.399633 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.400151 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.400838 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:39:58.401034 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:39:58.401108 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:39:58.403140 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:39:58.403142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:39:58.403142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:39:58.403152 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:39:58.404589 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.405000 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.405001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.405002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.405063 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:39:58.405275 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:39:58.405276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:39:58.405276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:39:58.405400 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:39:58.405478 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:39:58.405652 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:39:58.405734 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:39:58.408332 30645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464: Bootstrap starting.
I20260807 08:39:58.409122 30645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464: Neither blocks nor log segments found. Creating new log.
I20260807 08:39:58.409449 30645 log.cc:824] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464: Log is configured to *not* fsync() on all Append() calls
I20260807 08:39:58.410298 30645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464: No bootstrap required, opened a new log
I20260807 08:39:58.412304 30645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.412457 30645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:39:58.412504 30645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d410d3b1fd34d65af61e38a68e0c464, State: Initialized, Role: FOLLOWER
I20260807 08:39:58.412674 30645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.413038 30950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.413134 30645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:39:58.413196 30950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:39:58.415263 30970 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:39:58.415380 30970 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:39:58.418179 30543 master_runner.cc:263] No leader master found from master 0d410d3b1fd34d65af61e38a68e0c464
I20260807 08:39:58.418232 30543 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:39:58.449620 30579 tablet.cc:2369] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:39:58.491137 30950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:39:58.491305 30950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.491559 30950 leader_election.cc:288] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7b91f43c4dc409fb4c6bebd3f50bcd4 (127.10.207.61:41855), d57a1f63dc974e11aac9ec2f0d0053c7 (127.10.207.60:38339)
I20260807 08:39:58.491892 30842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d410d3b1fd34d65af61e38a68e0c464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" is_pre_election: true
I20260807 08:39:58.492100 30842 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d410d3b1fd34d65af61e38a68e0c464 in term 0.
I20260807 08:39:58.492317 30577 leader_election.cc:302] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d410d3b1fd34d65af61e38a68e0c464, d57a1f63dc974e11aac9ec2f0d0053c7; no voters:
I20260807 08:39:58.492475 30950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:39:58.492532 30950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:39:58.492564 30950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:58.493762 30950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.493891 30950 leader_election.cc:288] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [CANDIDATE]: Term 1 election: Requested vote from peers c7b91f43c4dc409fb4c6bebd3f50bcd4 (127.10.207.61:41855), d57a1f63dc974e11aac9ec2f0d0053c7 (127.10.207.60:38339)
I20260807 08:39:58.494114 30842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d410d3b1fd34d65af61e38a68e0c464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7"
I20260807 08:39:58.494196 30842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:58.495334 30842 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d410d3b1fd34d65af61e38a68e0c464 in term 1.
I20260807 08:39:58.495512 30577 leader_election.cc:302] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d410d3b1fd34d65af61e38a68e0c464, d57a1f63dc974e11aac9ec2f0d0053c7; no voters:
I20260807 08:39:58.495851 30728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d410d3b1fd34d65af61e38a68e0c464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" is_pre_election: true
I20260807 08:39:58.496016 30728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d410d3b1fd34d65af61e38a68e0c464 in term 0.
I20260807 08:39:58.496176 30727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d410d3b1fd34d65af61e38a68e0c464" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4"
I20260807 08:39:58.496269 30727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:39:58.497311 30727 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d410d3b1fd34d65af61e38a68e0c464 in term 1.
I20260807 08:39:58.497618 30950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:39:58.497764 30950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [term 1 LEADER]: Becoming Leader. State: Replica: 0d410d3b1fd34d65af61e38a68e0c464, State: Running, Role: LEADER
I20260807 08:39:58.497876 30950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } }
I20260807 08:39:58.498318 30950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d410d3b1fd34d65af61e38a68e0c464. Latest consensus state: current_term: 1 leader_uuid: "0d410d3b1fd34d65af61e38a68e0c464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.498405 30950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:58.498668 30977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:39:58.498912 30977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:39:58.501092 30842 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [term 1 FOLLOWER]: Refusing update from remote peer 0d410d3b1fd34d65af61e38a68e0c464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:58.501417 30950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:58.503015 30894 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d410d3b1fd34d65af61e38a68e0c464. Latest consensus state: current_term: 1 leader_uuid: "0d410d3b1fd34d65af61e38a68e0c464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.503149 30894 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:58.503520 30727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [term 1 FOLLOWER]: Refusing update from remote peer 0d410d3b1fd34d65af61e38a68e0c464: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:39:58.503784 30950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:39:58.505116 30931 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d410d3b1fd34d65af61e38a68e0c464. Latest consensus state: current_term: 1 leader_uuid: "0d410d3b1fd34d65af61e38a68e0c464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.505216 30931 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:58.507180 30978 mvcc.cc:205] Tried to move back new op lower bound from 7315832825858146304 to 7315832825847951360. Current Snapshot: MvccSnapshot[applied={T|T < 7315832825858146304}]
I20260807 08:39:58.507505 30950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d410d3b1fd34d65af61e38a68e0c464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.507603 30950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:58.507970 30931 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d410d3b1fd34d65af61e38a68e0c464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.508071 30931 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:58.508713 30981 mvcc.cc:205] Tried to move back new op lower bound from 7315832825858146304 to 7315832825847951360. Current Snapshot: MvccSnapshot[applied={T|T < 7315832825858146304}]
I20260807 08:39:58.509532 30950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d410d3b1fd34d65af61e38a68e0c464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.509614 30950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464 [sys.catalog]: This master's current role is: LEADER
I20260807 08:39:58.509948 30894 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d410d3b1fd34d65af61e38a68e0c464" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d410d3b1fd34d65af61e38a68e0c464" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38231 } } peers { permanent_uuid: "c7b91f43c4dc409fb4c6bebd3f50bcd4" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 41855 } } peers { permanent_uuid: "d57a1f63dc974e11aac9ec2f0d0053c7" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38339 } } }
I20260807 08:39:58.510033 30894 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:39:58.510687 30979 mvcc.cc:205] Tried to move back new op lower bound from 7315832825858146304 to 7315832825847951360. Current Snapshot: MvccSnapshot[applied={T|T < 7315832825858146304}]
I20260807 08:39:58.511261 30977 catalog_manager.cc:1347] Generated new cluster ID: 1f155afee9eb4471986a82d5a49cac56
I20260807 08:39:58.511312 30977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:39:58.523061 30977 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:39:58.524781 30977 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:39:58.530882 30977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d410d3b1fd34d65af61e38a68e0c464: Generated new TSK 0
I20260807 08:39:58.531062 30977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:39:59.356024 30914 catalog_manager.cc:1259] Loaded cluster ID: 1f155afee9eb4471986a82d5a49cac56
I20260807 08:39:59.356141 30914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: loading cluster ID for follower catalog manager: success
I20260807 08:39:59.357028 30914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: acquiring CA information for follower catalog manager: success
I20260807 08:39:59.357322 30914 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d57a1f63dc974e11aac9ec2f0d0053c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:39:59.392246 30946 catalog_manager.cc:1259] Loaded cluster ID: 1f155afee9eb4471986a82d5a49cac56
I20260807 08:39:59.392354 30946 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: loading cluster ID for follower catalog manager: success
I20260807 08:39:59.393219 30946 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: acquiring CA information for follower catalog manager: success
I20260807 08:39:59.393500 30946 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c7b91f43c4dc409fb4c6bebd3f50bcd4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:39:59.592120 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 30543
I20260807 08:39:59.602505 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 30656
I20260807 08:39:59.613535 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 30780
2026-08-07T08:39:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3205 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:39:59 dist-test-slave-h6n0 krb5kdc[31009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:39:59 dist-test-slave-h6n0 krb5kdc[31009](info): set up 2 sockets
Aug 07 08:39:59 dist-test-slave-h6n0 krb5kdc[31009](info): commencing operation
krb5kdc: starting...
W20260807 08:40:00.895292 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:40:00 dist-test-slave-h6n0 krb5kdc[31009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:40:01.034375 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41313
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:44823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:41313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:01.128997 31027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:01.129300 31027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:01.129341 31027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:01.130642 31027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:40:01.130712 31027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:01.130738 31027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:01.130759 31027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:40:01.130790 31027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:40:01.132750 31027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:44823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:41313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:01.133128 31027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:01.133420 31027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:01.139176 31041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:01.139277 31039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:01.139600 31027 server_base.cc:1034] running on GCE node
W20260807 08:40:01.139660 31038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:01.139986 31027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:01.140370 31027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:01.141556 31027 hybrid_clock.cc:648] HybridClock initialized: now 1786092001141532 us; error 61 us; skew 500 ppm
Aug 07 08:40:01 dist-test-slave-h6n0 krb5kdc[31009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092001, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:01.144625 31027 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:40:01.145175 31027 webserver.cc:460] Webserver started at http://127.10.207.62:43923/ using document root <none> and password file <none>
I20260807 08:40:01.145471 31027 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:01.145552 31027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:01.145756 31027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:01.147076 31027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "0b800d30659d4c11a5f191946b4f0955"
format_stamp: "Formatted at 2026-08-07 08:40:01 on dist-test-slave-h6n0"
server_key: "03f0a771aa1407731c027a19227c053e"
server_key_iv: "d393671748970b6575dfb303593d7548"
server_key_version: "encryptionkey@0"
I20260807 08:40:01.147509 31027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "0b800d30659d4c11a5f191946b4f0955"
format_stamp: "Formatted at 2026-08-07 08:40:01 on dist-test-slave-h6n0"
server_key: "03f0a771aa1407731c027a19227c053e"
server_key_iv: "d393671748970b6575dfb303593d7548"
server_key_version: "encryptionkey@0"
I20260807 08:40:01.149107 31027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:40:01.149981 31054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:01.150238 31027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:01.150353 31027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "0b800d30659d4c11a5f191946b4f0955"
format_stamp: "Formatted at 2026-08-07 08:40:01 on dist-test-slave-h6n0"
server_key: "03f0a771aa1407731c027a19227c053e"
server_key_iv: "d393671748970b6575dfb303593d7548"
server_key_version: "encryptionkey@0"
I20260807 08:40:01.150449 31027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:01.162063 31027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:01.164335 31027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:01.164451 31027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:01.168948 31027 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:41313
I20260807 08:40:01.169374 31027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:40:01.169397 31137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:41313 every 8 connection(s)
I20260807 08:40:01.170298 31138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:01.171733 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 31027
I20260807 08:40:01.171828 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:40:01.172070 11068 external_mini_cluster.cc:1428] Setting key 29da8d5b803e2d593628503308562f14
I20260807 08:40:01.173833 31138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955: Bootstrap starting.
I20260807 08:40:01.174682 31138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:01.175108 31138 log.cc:824] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:01.176050 31138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955: No bootstrap required, opened a new log
I20260807 08:40:01.178737 31138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b800d30659d4c11a5f191946b4f0955" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41313 } }
I20260807 08:40:01.178896 31138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:01.178920 31138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b800d30659d4c11a5f191946b4f0955, State: Initialized, Role: FOLLOWER
I20260807 08:40:01.179113 31138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b800d30659d4c11a5f191946b4f0955" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41313 } }
I20260807 08:40:01.179239 31138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:40:01.179287 31138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:40:01.179344 31138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:01.179986 31138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b800d30659d4c11a5f191946b4f0955" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41313 } }
I20260807 08:40:01.180150 31138 leader_election.cc:302] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b800d30659d4c11a5f191946b4f0955; no voters:
I20260807 08:40:01.180377 31138 leader_election.cc:288] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:40:01.180469 31144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:40:01.180686 31144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [term 1 LEADER]: Becoming Leader. State: Replica: 0b800d30659d4c11a5f191946b4f0955, State: Running, Role: LEADER
I20260807 08:40:01.180691 31138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 08:40:01 dist-test-slave-h6n0 krb5kdc[31009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:01.180828 31144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b800d30659d4c11a5f191946b4f0955" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41313 } }
I20260807 08:40:01.181231 31144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b800d30659d4c11a5f191946b4f0955. Latest consensus state: current_term: 1 leader_uuid: "0b800d30659d4c11a5f191946b4f0955" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b800d30659d4c11a5f191946b4f0955" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41313 } } }
I20260807 08:40:01.181337 31144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:01.181245 31146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b800d30659d4c11a5f191946b4f0955" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b800d30659d4c11a5f191946b4f0955" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41313 } } }
I20260807 08:40:01.181485 31146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955 [sys.catalog]: This master's current role is: LEADER
W20260807 08:40:01.183362 31170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:40:01.183424 31170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:40:01.183502 31156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:40:01.183665 31156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:40:01.185796 31156 catalog_manager.cc:1347] Generated new cluster ID: 8d77424eb80f456b9950ec5b24e36681
I20260807 08:40:01.185856 31156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:40:01.186622 31141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:01.172797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36012 (local address 127.10.207.62:41313)
0807 08:40:01.173036 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:01.173047 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:01.173087 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:40:01.173740 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:01.173751 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:01.173818 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:01.173993 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:01.174784 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:01.176258 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:01.177053 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:01.177765 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:01.177928 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:01.181523 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:01.181530 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:01.181530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:01.181544 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:01.184306 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:01.184777 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:01.184781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:01.184782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:01.184936 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:01.185217 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:01.185219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:01.185219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:01.185564 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:01.185730 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:01.186251 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 08:40:01.186394 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
I20260807 08:40:01.187744 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:40:01.199004 31156 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:40:01.199630 31156 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:40:01.205799 31156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b800d30659d4c11a5f191946b4f0955: Generated new TSK 0
I20260807 08:40:01.205997 31156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:40:01.242589 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 31027
2026-08-07T08:40:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1628 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:40:01 dist-test-slave-h6n0 krb5kdc[31178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:40:01 dist-test-slave-h6n0 krb5kdc[31178](info): set up 2 sockets
Aug 07 08:40:01 dist-test-slave-h6n0 krb5kdc[31178](info): commencing operation
krb5kdc: starting...
W20260807 08:40:02.532400 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:40:02 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:40:02.667666 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43323
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:02.760506 31205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:02.760759 31205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:02.760785 31205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:02.762003 31205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:40:02.762051 31205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:02.762069 31205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:02.762084 31205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:40:02.762136 31205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:40:02.763748 31205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:02.764017 31205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:02.764241 31205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:02.769524 31217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:02.769582 31212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:02.769524 31211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:02.770360 31205 server_base.cc:1034] running on GCE node
I20260807 08:40:02.770695 31205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:02.771036 31205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:02.772210 31205 hybrid_clock.cc:648] HybridClock initialized: now 1786092002772193 us; error 37 us; skew 500 ppm
Aug 07 08:40:02 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092002, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:02.775708 31205 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:40:02.776377 31205 webserver.cc:460] Webserver started at http://127.10.207.62:45785/ using document root <none> and password file <none>
I20260807 08:40:02.776711 31205 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:02.776793 31205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:02.776960 31205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:02.778116 31205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "8f439a83c322457d968c5d85a6cd21fc"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "ceeebebd5945ceea8b1a7ef5dc0b9ff9"
server_key_iv: "b32e9411ba839c5e59a9b947563578a3"
server_key_version: "encryptionkey@0"
I20260807 08:40:02.778517 31205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "8f439a83c322457d968c5d85a6cd21fc"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "ceeebebd5945ceea8b1a7ef5dc0b9ff9"
server_key_iv: "b32e9411ba839c5e59a9b947563578a3"
server_key_version: "encryptionkey@0"
I20260807 08:40:02.780037 31205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:40:02.781380 31228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:02.781610 31205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:02.781738 31205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "8f439a83c322457d968c5d85a6cd21fc"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "ceeebebd5945ceea8b1a7ef5dc0b9ff9"
server_key_iv: "b32e9411ba839c5e59a9b947563578a3"
server_key_version: "encryptionkey@0"
I20260807 08:40:02.781845 31205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:02.796139 31205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:02.798518 31205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:02.798808 31205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:02.803964 31205 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43323
I20260807 08:40:02.804010 31323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43323 every 8 connection(s)
I20260807 08:40:02.804458 31205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:40:02.805629 31324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:02.808003 31324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Bootstrap starting.
I20260807 08:40:02.808631 31324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:02.808935 31324 log.cc:824] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:02.809757 31324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: No bootstrap required, opened a new log
I20260807 08:40:02.811429 31324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:02.811571 31324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:02.811594 31324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f439a83c322457d968c5d85a6cd21fc, State: Initialized, Role: FOLLOWER
I20260807 08:40:02.811751 31324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:02.811817 31324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:40:02.811838 31324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:40:02.811890 31324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:02.812703 31324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:02.812820 31324 leader_election.cc:302] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f439a83c322457d968c5d85a6cd21fc; no voters:
I20260807 08:40:02.813017 31324 leader_election.cc:288] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:40:02.813078 31327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:40:02.813290 31324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:40:02.813321 31327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 LEADER]: Becoming Leader. State: Replica: 8f439a83c322457d968c5d85a6cd21fc, State: Running, Role: LEADER
I20260807 08:40:02.813452 31327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:02.813943 31330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f439a83c322457d968c5d85a6cd21fc. Latest consensus state: current_term: 1 leader_uuid: "8f439a83c322457d968c5d85a6cd21fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } } }
I20260807 08:40:02.814049 31330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:02.814240 31329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f439a83c322457d968c5d85a6cd21fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } } }
I20260807 08:40:02.814296 31338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:40:02.814409 31329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:02.814699 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 31205
I20260807 08:40:02.814867 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:40:02.815085 11068 external_mini_cluster.cc:1428] Setting key e4c49497736fe4c0a13054dff621b5d3
I20260807 08:40:02.815110 31338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:40:02.817287 31338 catalog_manager.cc:1347] Generated new cluster ID: 039b3f3ad2304300a0ec3e09864f3a70
I20260807 08:40:02.817353 31338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:40:02 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:02.828269 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:02.815889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.10.207.62:43323)
0807 08:40:02.816119 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:02.816130 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:02.816384 (+ 254us) server_negotiation.cc:407] Connection header received
0807 08:40:02.816545 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:02.816551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:02.816633 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:02.816793 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:02.817639 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:02.819142 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:02.820254 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:02.820938 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:02.821041 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:02.824109 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:02.824114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:02.824115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:02.824129 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:02.826410 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:02.826877 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:02.826880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:02.826881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:02.827018 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:02.827244 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:02.827245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:02.827245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:02.827575 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:02.827741 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:02.827849 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 08:40:02.828000 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":118,"threads_started":1}
I20260807 08:40:02.840323 31338 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:40:02.840931 31338 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:40:02.848477 31338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Generated new TSK 0
I20260807 08:40:02.848683 31338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:40:02.881855 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43323
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:02.972731 31365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:02.972946 31365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:02.972971 31365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:02.974097 31365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:02.974141 31365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:02.974170 31365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:40:02.975714 31365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:02.975984 31365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:02.976233 31365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:02.981698 31371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:02.981698 31375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:02.981698 31372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:02.981985 31365 server_base.cc:1034] running on GCE node
I20260807 08:40:02.982342 31365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:02.982687 31365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:02.983860 31365 hybrid_clock.cc:648] HybridClock initialized: now 1786092002983848 us; error 51 us; skew 500 ppm
Aug 07 08:40:02 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092002, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:02.986858 31365 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:40:02.987380 31365 webserver.cc:460] Webserver started at http://127.10.207.1:44321/ using document root <none> and password file <none>
I20260807 08:40:02.987633 31365 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:02.987684 31365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:02.987818 31365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:02.989023 31365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "81af2c9e0e2546f09ffaba8ecdc31161"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "a33aeb9852738bb783de281bb2288c07"
server_key_iv: "de14a13592cefc4309ed102eb6fb4795"
server_key_version: "encryptionkey@0"
I20260807 08:40:02.989428 31365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "81af2c9e0e2546f09ffaba8ecdc31161"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "a33aeb9852738bb783de281bb2288c07"
server_key_iv: "de14a13592cefc4309ed102eb6fb4795"
server_key_version: "encryptionkey@0"
I20260807 08:40:02.990968 31365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:40:02.991870 31385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:02.992110 31365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:40:02.992231 31365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "81af2c9e0e2546f09ffaba8ecdc31161"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "a33aeb9852738bb783de281bb2288c07"
server_key_iv: "de14a13592cefc4309ed102eb6fb4795"
server_key_version: "encryptionkey@0"
I20260807 08:40:02.992331 31365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:03.008929 31365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:03.011510 31365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:03.011686 31365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:03.011965 31365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:03.012328 31365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:03.012387 31365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.012434 31365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:03.012459 31365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.019999 31365 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:36729
I20260807 08:40:03.020071 31559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:36729 every 8 connection(s)
I20260807 08:40:03.020455 31365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:40:03 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092002, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:03.028688 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 31365
I20260807 08:40:03.028784 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:40:03.029060 11068 external_mini_cluster.cc:1428] Setting key 8910c1b27859a19da9f402319802a62d
I20260807 08:40:03.032891 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.021295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:40171 (local address 127.10.207.62:43323)
0807 08:40:03.021345 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.021347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.021581 (+ 234us) server_negotiation.cc:407] Connection header received
0807 08:40:03.022175 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.022177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.022189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.022243 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:03.023457 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.024172 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.025370 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.025604 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.025684 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.028076 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:03.028078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:03.028078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:03.028090 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:03.030887 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.031510 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.031511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.031512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.031592 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.031948 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.031950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.031950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.032107 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:03.032234 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.032625 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.032756 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:40:03.032922 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.021260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:40171)
0807 08:40:03.021439 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.021484 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.022068 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.022328 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.022332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.022989 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:03.023289 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.023300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.024305 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.024307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.025280 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.025283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.025693 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.025694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.025818 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.026362 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:03.026370 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:03.027867 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:03.030987 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:03.030998 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:03.031001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:03.031397 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:03.031676 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:03.031677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:03.031678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:03.031840 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:03.032209 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:03.032218 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:03.032442 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.032569 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.032724 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260807 08:40:03.033587 31560 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43323
I20260807 08:40:03.033733 31560 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:03.033962 31560 heartbeater.cc:511] Master 127.10.207.62:43323 requested a full tablet report, sending...
I20260807 08:40:03.034716 31262 ts_manager.cc:194] Registered new tserver with Master: 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1:36729)
I20260807 08:40:03.035408 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:40171
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:40:03.059940 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43323
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:03.148276 31568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:03.148478 31568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:03.148502 31568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:03.149614 31568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:03.149657 31568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:03.149683 31568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:40:03.151135 31568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:03.151353 31568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:03.151602 31568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:03.156874 31583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:03.156930 31581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:03.156890 31580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:03.157032 31568 server_base.cc:1034] running on GCE node
I20260807 08:40:03.157447 31568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:03.157719 31568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:03.158890 31568 hybrid_clock.cc:648] HybridClock initialized: now 1786092003158877 us; error 31 us; skew 500 ppm
Aug 07 08:40:03 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092003, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:03.161949 31568 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:40:03.162441 31568 webserver.cc:460] Webserver started at http://127.10.207.2:44669/ using document root <none> and password file <none>
I20260807 08:40:03.162734 31568 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:03.162806 31568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:03.162963 31568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:03.163966 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "f938ccc0935b495d900cd3e2a481c903"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "e690ac2b44b5a9691720a4732d3b8e1d"
server_key_iv: "42ea579cc803740e1f302a4c72fbd1da"
server_key_version: "encryptionkey@0"
I20260807 08:40:03.164332 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "f938ccc0935b495d900cd3e2a481c903"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "e690ac2b44b5a9691720a4732d3b8e1d"
server_key_iv: "42ea579cc803740e1f302a4c72fbd1da"
server_key_version: "encryptionkey@0"
I20260807 08:40:03.165772 31568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:40:03.166698 31593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.166932 31568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:03.167033 31568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "f938ccc0935b495d900cd3e2a481c903"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "e690ac2b44b5a9691720a4732d3b8e1d"
server_key_iv: "42ea579cc803740e1f302a4c72fbd1da"
server_key_version: "encryptionkey@0"
I20260807 08:40:03.167121 31568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:03.177605 31568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:03.180106 31568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:03.180267 31568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:03.180510 31568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:03.180876 31568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:03.180938 31568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.180979 31568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:03.181006 31568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.188279 31568 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:41135
I20260807 08:40:03.188336 31785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:41135 every 8 connection(s)
I20260807 08:40:03.188627 31568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:40:03 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092003, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:03.196892 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 31568
I20260807 08:40:03.197046 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:40:03.197302 11068 external_mini_cluster.cc:1428] Setting key ccba86016e9f83433d0a8e590711a437
I20260807 08:40:03.199596 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.189391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:52009 (local address 127.10.207.62:43323)
0807 08:40:03.189496 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.189499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.189718 (+ 219us) server_negotiation.cc:407] Connection header received
0807 08:40:03.190376 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.190378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.190393 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.190458 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:03.191627 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.192214 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.193279 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.193511 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.193598 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.196088 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:03.196090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:03.196090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:03.196099 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:03.197912 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.198435 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.198436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.198437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.198500 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.198805 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.198806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.198807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.198979 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:03.199066 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.199362 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.199465 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:40:03.199635 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.189294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:52009)
0807 08:40:03.189524 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.189582 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.190251 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.190570 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.190575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.191268 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:03.191503 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.191508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.192320 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.192322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.193186 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.193189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.193618 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.193619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.193749 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.194250 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:03.194259 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:03.195970 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:03.198042 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:03.198050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:03.198052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:03.198347 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:03.198619 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:03.198619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:03.198620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:03.198728 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:03.199064 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:03.199069 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:03.199222 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.199312 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.199448 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260807 08:40:03.200372 31787 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43323
I20260807 08:40:03.200498 31787 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:03.200757 31787 heartbeater.cc:511] Master 127.10.207.62:43323 requested a full tablet report, sending...
I20260807 08:40:03.201283 31262 ts_manager.cc:194] Registered new tserver with Master: f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135)
I20260807 08:40:03.201706 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:52009
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:40:03.227275 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43323
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:03.320430 31800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:03.320660 31800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:03.320685 31800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:03.321866 31800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:03.321913 31800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:03.321941 31800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:40:03.323599 31800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:03.323872 31800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:03.324151 31800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:03.329849 31806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:03.329860 31809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:03.329849 31807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:03.330411 31800 server_base.cc:1034] running on GCE node
I20260807 08:40:03.330713 31800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:03.330972 31800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:03.332135 31800 hybrid_clock.cc:648] HybridClock initialized: now 1786092003332128 us; error 34 us; skew 500 ppm
Aug 07 08:40:03 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092003, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:03.335110 31800 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:40:03.335587 31800 webserver.cc:460] Webserver started at http://127.10.207.3:36471/ using document root <none> and password file <none>
I20260807 08:40:03.335834 31800 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:03.335882 31800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:03.335983 31800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:03.337105 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "72d05aaf06b94642b5a2bdc4088fd078"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "62f3539d6aa12f36107e67f6411f9a93"
server_key_iv: "74c876f8ca7a41aec6800c580e24a501"
server_key_version: "encryptionkey@0"
I20260807 08:40:03.337436 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "72d05aaf06b94642b5a2bdc4088fd078"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "62f3539d6aa12f36107e67f6411f9a93"
server_key_iv: "74c876f8ca7a41aec6800c580e24a501"
server_key_version: "encryptionkey@0"
I20260807 08:40:03.338914 31800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:40:03.339915 31820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.340173 31800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:03.340317 31800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "72d05aaf06b94642b5a2bdc4088fd078"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "62f3539d6aa12f36107e67f6411f9a93"
server_key_iv: "74c876f8ca7a41aec6800c580e24a501"
server_key_version: "encryptionkey@0"
I20260807 08:40:03.340402 31800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:03.366886 31800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:03.369237 31800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:03.369414 31800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:03.369751 31800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:03.370169 31800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:03.370227 31800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.370275 31800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:03.370302 31800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.377326 31800 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:40945
I20260807 08:40:03.377449 31999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:40945 every 8 connection(s)
I20260807 08:40:03.377748 31800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:40:03.384327 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 31800
Aug 07 08:40:03 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092003, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:03.384459 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:40:03.384661 11068 external_mini_cluster.cc:1428] Setting key 48d979b7408b051c3a544ddc6b35b0b9
I20260807 08:40:03.387991 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.378705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:33395 (local address 127.10.207.62:43323)
0807 08:40:03.378763 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.378765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.379088 (+ 323us) server_negotiation.cc:407] Connection header received
0807 08:40:03.379666 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.379667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.379678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.379746 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:03.380771 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.381276 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.382359 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.382601 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.382668 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.385049 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:03.385051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:03.385052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:03.385061 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:03.386691 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.387148 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.387149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.387149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.387203 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.387416 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.387416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.387416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.387527 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:03.387577 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.387809 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.387909 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:40:03.388173 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.378720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:33395)
0807 08:40:03.378926 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.378975 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.379564 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.379809 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.379813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.380439 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:03.380688 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.380694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.381372 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.381373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.382277 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.382280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.382689 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.382690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.382801 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.383258 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:03.383266 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:03.384939 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:03.386788 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:03.386795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:03.386798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:03.387068 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:03.387262 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:03.387263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:03.387263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:03.387355 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:03.387575 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:03.387578 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:03.387693 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.387785 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.387924 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
I20260807 08:40:03.388705 32002 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43323
I20260807 08:40:03.388813 32002 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:03.388993 32002 heartbeater.cc:511] Master 127.10.207.62:43323 requested a full tablet report, sending...
I20260807 08:40:03.389405 31262 ts_manager.cc:194] Registered new tserver with Master: 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
I20260807 08:40:03.389784 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:33395
I20260807 08:40:03.397095 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:40:03.405732 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.398704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.10.207.62:43323)
0807 08:40:03.398785 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.398788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.398914 (+ 126us) server_negotiation.cc:407] Connection header received
0807 08:40:03.398966 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.398967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.398976 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.399031 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:03.399670 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.400186 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.400975 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.401175 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.401242 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.402423 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:03.402424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:03.402424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:03.402431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:03.404369 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.404807 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.404809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.404809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.404878 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:03.405100 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:03.405101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:03.405101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:03.405268 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:03.405398 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.405519 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.405611 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:40:03.407651 31262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:40:03.409650 31262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:40:03.417052 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.412567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:40945 (local address 127.0.0.1:59846)
0807 08:40:03.412806 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.412856 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.412922 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.413258 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.413262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.413265 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:03.413347 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.413349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.414455 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.414456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.415672 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.415675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.416694 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.416695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.416813 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.416831 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.416920 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.416974 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
I20260807 08:40:03.417106 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.412712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.10.207.3:40945)
0807 08:40:03.412928 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.412934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.412948 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:40:03.413006 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.413008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.413027 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.413165 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:03.413441 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.414279 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.415763 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.416578 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.416667 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.416735 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.416952 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.416999 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260807 08:40:03.417948 31904 tablet_service.cc:1467] Processing CreateTablet for tablet 5e5c974bca70474ab5cd05ce25a21886 (DEFAULT_TABLE table=test-table [id=1999aa0594974825b41f5ad601007b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:03.418352 31904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e5c974bca70474ab5cd05ce25a21886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:03.418708 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.412811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36729 (local address 127.0.0.1:40304)
0807 08:40:03.414407 (+ 1596us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.414437 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.414496 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.414761 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.414762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.414765 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:03.414831 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.414833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.416133 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.416135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.417380 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.417406 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.418392 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.418393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.418476 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.418481 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.418575 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.418633 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1576,"thread_start_us":100,"threads_started":1}
I20260807 08:40:03.418716 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.413058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.10.207.1:36729)
0807 08:40:03.413378 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.413383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.414594 (+ 1211us) server_negotiation.cc:407] Connection header received
0807 08:40:03.414655 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.414657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.414670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.414762 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:03.414883 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.415960 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.417498 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.418296 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.418387 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.418440 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.418580 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.418628 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":127,"threads_started":1}
I20260807 08:40:03.420334 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.412845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.10.207.2:41135)
0807 08:40:03.416184 (+ 3339us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.416189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.416199 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:40:03.416241 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.416243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.416259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.416356 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:03.416570 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.417903 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.419240 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.419924 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.419995 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.420034 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.420226 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.420268 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3294,"thread_start_us":116,"threads_started":1}
I20260807 08:40:03.420359 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.412616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41135 (local address 127.0.0.1:35192)
0807 08:40:03.413066 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.413079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.413181 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.416387 (+ 3206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.416388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.416391 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:03.416452 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.416455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.418010 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.418012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.419111 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.419115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.420037 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.420038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.420114 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.420118 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.420202 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.420247 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":124,"threads_started":1}
I20260807 08:40:03.421244 31676 tablet_service.cc:1467] Processing CreateTablet for tablet 5e5c974bca70474ab5cd05ce25a21886 (DEFAULT_TABLE table=test-table [id=1999aa0594974825b41f5ad601007b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:03.421629 31676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e5c974bca70474ab5cd05ce25a21886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:03.421881 32025 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Bootstrap starting.
I20260807 08:40:03.422390 31460 tablet_service.cc:1467] Processing CreateTablet for tablet 5e5c974bca70474ab5cd05ce25a21886 (DEFAULT_TABLE table=test-table [id=1999aa0594974825b41f5ad601007b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:03.422777 31460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e5c974bca70474ab5cd05ce25a21886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:03.422910 32025 tablet_bootstrap.cc:654] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:03.423575 32025 log.cc:824] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:03.424605 32025 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: No bootstrap required, opened a new log
I20260807 08:40:03.424697 32025 ts_tablet_manager.cc:1397] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:40:03.425918 32029 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Bootstrap starting.
I20260807 08:40:03.426347 32030 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap starting.
I20260807 08:40:03.426837 32029 tablet_bootstrap.cc:654] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:03.427168 32030 tablet_bootstrap.cc:654] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:03.427073 32025 raft_consensus.cc:348] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.427253 32025 raft_consensus.cc:374] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:03.427311 32025 raft_consensus.cc:729] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72d05aaf06b94642b5a2bdc4088fd078, State: Initialized, Role: FOLLOWER
I20260807 08:40:03.427405 32029 log.cc:824] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:03.427464 32025 consensus_queue.cc:260] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.427826 32025 ts_tablet_manager.cc:1428] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 08:40:03.427856 32002 heartbeater.cc:503] Master 127.10.207.62:43323 was elected leader, sending a full tablet report...
I20260807 08:40:03.427865 32030 log.cc:824] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:03.428721 32029 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: No bootstrap required, opened a new log
I20260807 08:40:03.428761 32030 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: No bootstrap required, opened a new log
I20260807 08:40:03.428802 32029 ts_tablet_manager.cc:1397] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:40:03.428838 32030 ts_tablet_manager.cc:1397] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:40:03.430379 32030 raft_consensus.cc:348] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.430572 32030 raft_consensus.cc:374] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:03.430398 32029 raft_consensus.cc:348] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.430625 32030 raft_consensus.cc:729] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81af2c9e0e2546f09ffaba8ecdc31161, State: Initialized, Role: FOLLOWER
I20260807 08:40:03.430629 32029 raft_consensus.cc:374] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:03.430675 32029 raft_consensus.cc:729] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f938ccc0935b495d900cd3e2a481c903, State: Initialized, Role: FOLLOWER
I20260807 08:40:03.430747 32030 consensus_queue.cc:260] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.430795 32029 consensus_queue.cc:260] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.431042 32030 ts_tablet_manager.cc:1428] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:40:03.431114 32029 ts_tablet_manager.cc:1428] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:40:03.431272 31560 heartbeater.cc:503] Master 127.10.207.62:43323 was elected leader, sending a full tablet report...
I20260807 08:40:03.431648 31787 heartbeater.cc:503] Master 127.10.207.62:43323 was elected leader, sending a full tablet report...
W20260807 08:40:03.439257 31792 tablet.cc:2369] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:40:03.521373 31561 tablet.cc:2369] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:40:03.628477 32003 tablet.cc:2369] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:40:03.742714 32032 raft_consensus.cc:482] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:03.742882 32032 raft_consensus.cc:504] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.743403 32032 leader_election.cc:288] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1:36729)
I20260807 08:40:03.747432 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.743605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41135 (local address 127.10.207.3:40383)
0807 08:40:03.743678 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.743692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.743749 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.743958 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.743959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.743962 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:03.744049 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.744051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.745007 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.745009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.746144 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.746147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.747071 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.747072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.747191 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.747197 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.747291 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.747348 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 08:40:03.747433 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.743701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:40383 (local address 127.10.207.2:41135)
0807 08:40:03.743768 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.743770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.743789 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:40:03.743837 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.743838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.743854 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.743937 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:03.744153 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.744839 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.746265 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.746864 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.746924 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.746934 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.747315 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.747363 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:40:03.747939 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.743732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36729 (local address 127.10.207.3:33817)
0807 08:40:03.744134 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.744147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:03.744223 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:03.744407 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:03.744408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:03.744410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:03.744495 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.744497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.745626 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.745627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.746666 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:03.746668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.747631 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.747632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:03.747722 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.747726 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:40:03.747809 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:40:03.747860 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":179,"threads_started":1}
I20260807 08:40:03.747982 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.743926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:33817 (local address 127.10.207.1:36729)
0807 08:40:03.744035 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.744043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.744254 (+ 211us) server_negotiation.cc:407] Connection header received
0807 08:40:03.744306 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.744307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.744318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.744389 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:03.744612 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.745497 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.746796 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.747467 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.747569 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.747582 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.747835 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.747902 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 08:40:03.747920 31715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e5c974bca70474ab5cd05ce25a21886" candidate_uuid: "72d05aaf06b94642b5a2bdc4088fd078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f938ccc0935b495d900cd3e2a481c903" is_pre_election: true
I20260807 08:40:03.748143 31715 raft_consensus.cc:2454] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72d05aaf06b94642b5a2bdc4088fd078 in term 0.
I20260807 08:40:03.748476 31823 leader_election.cc:302] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d05aaf06b94642b5a2bdc4088fd078, f938ccc0935b495d900cd3e2a481c903; no voters:
I20260807 08:40:03.748660 32032 raft_consensus.cc:2781] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:40:03.748611 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e5c974bca70474ab5cd05ce25a21886" candidate_uuid: "72d05aaf06b94642b5a2bdc4088fd078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" is_pre_election: true
I20260807 08:40:03.748730 32032 raft_consensus.cc:482] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:40:03.748764 32032 raft_consensus.cc:3037] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:03.748788 31495 raft_consensus.cc:2454] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72d05aaf06b94642b5a2bdc4088fd078 in term 0.
I20260807 08:40:03.749598 32032 raft_consensus.cc:504] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.749799 32032 leader_election.cc:288] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [CANDIDATE]: Term 1 election: Requested vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1:36729)
I20260807 08:40:03.750006 31715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e5c974bca70474ab5cd05ce25a21886" candidate_uuid: "72d05aaf06b94642b5a2bdc4088fd078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f938ccc0935b495d900cd3e2a481c903"
I20260807 08:40:03.750011 31495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e5c974bca70474ab5cd05ce25a21886" candidate_uuid: "72d05aaf06b94642b5a2bdc4088fd078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81af2c9e0e2546f09ffaba8ecdc31161"
I20260807 08:40:03.750131 31715 raft_consensus.cc:3037] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:03.750152 31495 raft_consensus.cc:3037] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:03.751031 31495 raft_consensus.cc:2454] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72d05aaf06b94642b5a2bdc4088fd078 in term 1.
I20260807 08:40:03.751215 31715 raft_consensus.cc:2454] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72d05aaf06b94642b5a2bdc4088fd078 in term 1.
I20260807 08:40:03.751446 31822 leader_election.cc:302] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d05aaf06b94642b5a2bdc4088fd078, 81af2c9e0e2546f09ffaba8ecdc31161; no voters:
I20260807 08:40:03.751585 32032 raft_consensus.cc:2781] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:40:03.751776 32032 raft_consensus.cc:686] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 LEADER]: Becoming Leader. State: Replica: 72d05aaf06b94642b5a2bdc4088fd078, State: Running, Role: LEADER
I20260807 08:40:03.751895 32032 consensus_queue.cc:237] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:03.752941 31262 catalog_manager.cc:5495] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3). New cstate: current_term: 1 leader_uuid: "72d05aaf06b94642b5a2bdc4088fd078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } health_report { overall_health: UNKNOWN } } }
I20260807 08:40:03.771471 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.768874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.10.207.3:40945)
0807 08:40:03.768957 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:03.768959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:03.768968 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:40:03.769007 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:03.769008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:03.769027 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:03.769115 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:40:03.769276 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.769783 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.770692 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:03.770895 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:03.770955 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:03.771145 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:40:03.771256 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:40:03.771328 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:03.771372 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 08:40:03.771391 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:40:03.773597 31495 raft_consensus.cc:1261] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Refusing update from remote peer 72d05aaf06b94642b5a2bdc4088fd078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:03.773674 31715 raft_consensus.cc:1261] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Refusing update from remote peer 72d05aaf06b94642b5a2bdc4088fd078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:03.774065 32041 consensus_queue.cc:1035] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:03.774197 32032 consensus_queue.cc:1035] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:03.777400 32046 mvcc.cc:205] Tried to move back new op lower bound from 7315832847453564928 to 7315832847368454144. Current Snapshot: MvccSnapshot[applied={T|T < 7315832847453564928}]
I20260807 08:40:03.777937 32047 mvcc.cc:205] Tried to move back new op lower bound from 7315832847453564928 to 7315832847368454144. Current Snapshot: MvccSnapshot[applied={T|T < 7315832847453564928}]
I20260807 08:40:03.780402 31262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:03.780496 31262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:03.781347 31262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Sending DeleteTablet for 3 replicas of tablet 5e5c974bca70474ab5cd05ce25a21886
I20260807 08:40:03.781706 31904 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e5c974bca70474ab5cd05ce25a21886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:03 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:59846
I20260807 08:40:03.781723 31676 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e5c974bca70474ab5cd05ce25a21886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:03 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:35192
I20260807 08:40:03.781747 31460 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e5c974bca70474ab5cd05ce25a21886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:03 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:40304
I20260807 08:40:03.782089 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 31365
I20260807 08:40:03.782101 32067 tablet_replica.cc:330] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: stopping tablet replica
I20260807 08:40:03.782212 32067 raft_consensus.cc:2229] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:03.782335 32067 raft_consensus.cc:2258] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:03.782204 32065 tablet_replica.cc:330] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: stopping tablet replica
I20260807 08:40:03.793644 32067 ts_tablet_manager.cc:1905] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:03.793668 32065 raft_consensus.cc:2229] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:40:03.793901 32065 raft_consensus.cc:2258] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:03.794659 32065 ts_tablet_manager.cc:1905] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:03.795949 32067 ts_tablet_manager.cc:1918] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:03.796042 32067 log.cc:1196] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/5e5c974bca70474ab5cd05ce25a21886
I20260807 08:40:03.796311 32067 ts_tablet_manager.cc:1939] T 5e5c974bca70474ab5cd05ce25a21886 P f938ccc0935b495d900cd3e2a481c903: Deleting consensus metadata
I20260807 08:40:03.796785 31233 catalog_manager.cc:4841] TS f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135): tablet 5e5c974bca70474ab5cd05ce25a21886 (table test-table [id=1999aa0594974825b41f5ad601007b8f]) successfully deleted
I20260807 08:40:03.797116 32065 ts_tablet_manager.cc:1918] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:03.797209 32065 log.cc:1196] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/5e5c974bca70474ab5cd05ce25a21886
I20260807 08:40:03.797513 32065 ts_tablet_manager.cc:1939] T 5e5c974bca70474ab5cd05ce25a21886 P 72d05aaf06b94642b5a2bdc4088fd078: Deleting consensus metadata
I20260807 08:40:03.798024 31231 catalog_manager.cc:4841] TS 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945): tablet 5e5c974bca70474ab5cd05ce25a21886 (table test-table [id=1999aa0594974825b41f5ad601007b8f]) successfully deleted
W20260807 08:40:03.799876 31231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:36729 (error 108)
I20260807 08:40:03.800177 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 31568
I20260807 08:40:03.800372 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:03.800253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36729 (local address 127.0.0.1:40306)
0807 08:40:03.800301 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:03.800336 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 08:40:03.800485 31231 catalog_manager.cc:4584] TS 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1:36729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e5c974bca70474ab5cd05ce25a21886: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36729: connect: Connection refused (error 111)
I20260807 08:40:03.811630 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 31800
I20260807 08:40:03.823381 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 31205
I20260807 08:40:03.836704 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43323
--webserver_interface=127.10.207.62
--webserver_port=45785
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:03.929062 32069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:03.929267 32069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:03.929294 32069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:03.930433 32069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:40:03.930480 32069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:03.930495 32069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:03.930509 32069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:40:03.930585 32069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:40:03.932052 32069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=45785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:03.932296 32069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:03.932559 32069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:03.943142 32083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:03.943192 32077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:03.943521 32069 server_base.cc:1034] running on GCE node
W20260807 08:40:03.943672 32079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:03.944089 32069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:03.944514 32069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:03.945704 32069 hybrid_clock.cc:648] HybridClock initialized: now 1786092003945693 us; error 31 us; skew 500 ppm
Aug 07 08:40:03 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092003, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:03.948604 32069 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:40:03.949129 32069 webserver.cc:460] Webserver started at http://127.10.207.62:45785/ using document root <none> and password file <none>
I20260807 08:40:03.949397 32069 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:03.949486 32069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:03.951478 32069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:03.952410 32092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:03.952664 32069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:03.952791 32069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "8f439a83c322457d968c5d85a6cd21fc"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "ceeebebd5945ceea8b1a7ef5dc0b9ff9"
server_key_iv: "b32e9411ba839c5e59a9b947563578a3"
server_key_version: "encryptionkey@0"
I20260807 08:40:03.953339 32069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:03.976099 32069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:03.978205 32069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:03.978381 32069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:03.983898 32069 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43323
I20260807 08:40:03.983964 32180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43323 every 8 connection(s)
I20260807 08:40:03.984424 32069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:40:03.986338 32181 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:40:03.987202 32181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Bootstrap starting.
I20260807 08:40:03.993899 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 32069
I20260807 08:40:03.994278 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:36729
--local_ip_for_outbound_sockets=127.10.207.1
--tserver_master_addrs=127.10.207.62:43323
--webserver_port=44321
--webserver_interface=127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:03.996593 32181 log.cc:824] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:03.999455 32181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:40:03.999809 32181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Bootstrap complete.
I20260807 08:40:04.003947 32181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.004379 32181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f439a83c322457d968c5d85a6cd21fc, State: Initialized, Role: FOLLOWER
I20260807 08:40:04.004608 32181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.004714 32181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:40:04.004771 32181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:40:04.004822 32181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:40:04.006430 32181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.006583 32181 leader_election.cc:302] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f439a83c322457d968c5d85a6cd21fc; no voters:
I20260807 08:40:04.006778 32181 leader_election.cc:288] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:40:04.006920 32188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:40:04.007078 32181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:40:04.007222 32188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 LEADER]: Becoming Leader. State: Replica: 8f439a83c322457d968c5d85a6cd21fc, State: Running, Role: LEADER
I20260807 08:40:04.007364 32188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.007787 32189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8f439a83c322457d968c5d85a6cd21fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } } }
I20260807 08:40:04.007880 32189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:04.008065 32190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f439a83c322457d968c5d85a6cd21fc. Latest consensus state: current_term: 2 leader_uuid: "8f439a83c322457d968c5d85a6cd21fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } } }
I20260807 08:40:04.008126 32190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:04.008394 32198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:40:04.009428 32198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:40:04.009593 32198 catalog_manager.cc:1259] Loaded cluster ID: 039b3f3ad2304300a0ec3e09864f3a70
I20260807 08:40:04.009897 32198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:40:04.010239 32198 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 08:40:04.014909 32198 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 08:40:04.017866 32198 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:40:04.018119 32198 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Loaded TSK: 0
I20260807 08:40:04.018375 32198 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:40:04.088153 32185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:04.088406 32185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:04.088466 32185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:04.089668 32185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:04.089735 32185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:04.089782 32185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:40:04.091364 32185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:36729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=44321
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:04.091655 32185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:04.091883 32185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:04.097918 32217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.097926 32222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.098021 32219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:04.099036 32185 server_base.cc:1034] running on GCE node
I20260807 08:40:04.099282 32185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:04.099555 32185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:04.100739 32185 hybrid_clock.cc:648] HybridClock initialized: now 1786092004100712 us; error 48 us; skew 500 ppm
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:04.104620 32185 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:40:04.105243 32185 webserver.cc:460] Webserver started at http://127.10.207.1:44321/ using document root <none> and password file <none>
I20260807 08:40:04.105647 32185 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:04.105793 32185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:04.107694 32185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:04.108566 32230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.108860 32185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:04.108978 32185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "81af2c9e0e2546f09ffaba8ecdc31161"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "a33aeb9852738bb783de281bb2288c07"
server_key_iv: "de14a13592cefc4309ed102eb6fb4795"
server_key_version: "encryptionkey@0"
I20260807 08:40:04.109568 32185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:04.125666 32185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:04.128162 32185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:04.128332 32185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:04.128597 32185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:04.129112 32248 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:40:04.130621 32185 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:40:04.130685 32185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:40:04.130714 32185 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:40:04.131798 32185 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:40:04.131871 32185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:04.131896 32248 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap starting.
I20260807 08:40:04.142470 32185 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:36729
I20260807 08:40:04.143044 32185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:40:04.145766 32248 log.cc:824] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:04.146099 32423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:36729 every 8 connection(s)
I20260807 08:40:04.148698 32248 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:40:04.149029 32248 tablet_bootstrap.cc:492] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap complete.
I20260807 08:40:04.150280 32248 ts_tablet_manager.cc:1397] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20260807 08:40:04.151576 32248 raft_consensus.cc:348] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.151842 32248 raft_consensus.cc:729] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81af2c9e0e2546f09ffaba8ecdc31161, State: Initialized, Role: FOLLOWER
I20260807 08:40:04.152019 32248 consensus_queue.cc:260] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.152323 32248 ts_tablet_manager.cc:1428] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:04.152832 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 32185
I20260807 08:40:04.153206 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:41135
--local_ip_for_outbound_sockets=127.10.207.2
--tserver_master_addrs=127.10.207.62:43323
--webserver_port=44669
--webserver_interface=127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:04.157769 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.143956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:34145)
0807 08:40:04.144166 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.144209 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.144964 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.145579 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.145584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.145967 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:04.146254 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.146260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.148069 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.148071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.149471 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.149476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.150420 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.150422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.150646 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.151376 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:04.151387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:04.153193 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:04.155593 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.155602 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.155604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.155868 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.156285 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.156286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.156286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.156415 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.157154 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:04.157161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:04.157336 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.157435 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.157599 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":129,"threads_started":1}
I20260807 08:40:04.158021 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.144279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:34145 (local address 127.10.207.62:43323)
0807 08:40:04.144517 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.144527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.144574 (+ 47us) server_negotiation.cc:407] Connection header received
0807 08:40:04.145244 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.145252 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.145314 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.145504 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:04.146411 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.147906 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.149591 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.150274 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.150465 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.153487 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:04.153491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:04.153491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:04.153502 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:04.155426 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.155993 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.155996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.155996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.156124 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.156544 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.156546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.156547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.156985 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:04.157165 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.157519 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.157796 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
I20260807 08:40:04.158814 32425 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43323
I20260807 08:40:04.158914 32425 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:04.159137 32425 heartbeater.cc:511] Master 127.10.207.62:43323 requested a full tablet report, sending...
I20260807 08:40:04.160055 32116 ts_manager.cc:194] Registered new tserver with Master: 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1:36729)
I20260807 08:40:04.161396 32116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:34145
I20260807 08:40:04.162460 32425 heartbeater.cc:503] Master 127.10.207.62:43323 was elected leader, sending a full tablet report...
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092003, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:40:04.170441 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.160805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.10.207.1:36729)
0807 08:40:04.161014 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.161019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.161137 (+ 118us) server_negotiation.cc:407] Connection header received
0807 08:40:04.161273 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.161275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.161288 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.161362 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:04.162022 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.163261 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.164090 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.164600 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.164829 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.167180 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:04.167188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:04.167188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:04.167195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:04.168904 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.169422 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.169424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.169424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.169481 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.169719 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.169720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.169720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.169919 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:04.170008 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.170276 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.170371 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260807 08:40:04.170473 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.160767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36729 (local address 127.0.0.1:40316)
0807 08:40:04.161018 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.161061 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.161159 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.161383 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.161388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.161841 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:04.161928 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.161931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.163362 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.163363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.163992 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.163994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.164720 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.164721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.164802 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.165389 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:04.165395 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:04.167041 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:04.169004 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.169012 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.169014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.169322 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.169561 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.169562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.169562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.169640 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.170012 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:04.170019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:04.170083 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.170203 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.170361 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":109,"threads_started":1}
I20260807 08:40:04.171025 32311 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e5c974bca70474ab5cd05ce25a21886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:03 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:40316
I20260807 08:40:04.171671 32439 tablet_replica.cc:330] stopping tablet replica
I20260807 08:40:04.171806 32439 raft_consensus.cc:2229] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:04.171897 32439 raft_consensus.cc:2258] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:04.172271 32439 ts_tablet_manager.cc:1905] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:04.174397 32439 ts_tablet_manager.cc:1918] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:04.174490 32439 log.cc:1196] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/5e5c974bca70474ab5cd05ce25a21886
I20260807 08:40:04.174768 32439 ts_tablet_manager.cc:1939] T 5e5c974bca70474ab5cd05ce25a21886 P 81af2c9e0e2546f09ffaba8ecdc31161: Deleting consensus metadata
I20260807 08:40:04.175297 32094 catalog_manager.cc:4841] TS 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1:36729): tablet 5e5c974bca70474ab5cd05ce25a21886 (table test-table [id=1999aa0594974825b41f5ad601007b8f]) successfully deleted
W20260807 08:40:04.247673 32433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:04.247905 32433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:04.247929 32433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:04.249109 32433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:04.249154 32433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:04.249181 32433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:40:04.250715 32433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:41135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=44669
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:04.250989 32433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:04.251209 32433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:04.257517 32446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.257557 32447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.257536 32449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:04.257716 32433 server_base.cc:1034] running on GCE node
I20260807 08:40:04.257929 32433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:04.258206 32433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:04.259395 32433 hybrid_clock.cc:648] HybridClock initialized: now 1786092004259368 us; error 46 us; skew 500 ppm
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:04.262127 32433 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:40:04.262686 32433 webserver.cc:460] Webserver started at http://127.10.207.2:44669/ using document root <none> and password file <none>
I20260807 08:40:04.262941 32433 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:04.263026 32433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:04.264525 32433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:40:04.265195 32461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.265426 32433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:04.265538 32433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "f938ccc0935b495d900cd3e2a481c903"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "e690ac2b44b5a9691720a4732d3b8e1d"
server_key_iv: "42ea579cc803740e1f302a4c72fbd1da"
server_key_version: "encryptionkey@0"
I20260807 08:40:04.265940 32433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:04.311535 32433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:04.313949 32433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:04.314124 32433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:04.314416 32433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:04.314815 32433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:04.314873 32433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.314921 32433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:04.314947 32433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.322410 32433 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:41135
I20260807 08:40:04.322528 32652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:41135 every 8 connection(s)
I20260807 08:40:04.322944 32433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:04.332569 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 32433
I20260807 08:40:04.332691 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.323859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:43903 (local address 127.10.207.62:43323)
0807 08:40:04.323945 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.323948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.324254 (+ 306us) server_negotiation.cc:407] Connection header received
0807 08:40:04.324792 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.324793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.324805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.324868 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:04.325915 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.326467 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.327500 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.327705 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.327770 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.329886 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:04.329888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:04.329888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:04.329894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:04.331321 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.331714 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.331714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.331715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.331774 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.332023 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.332023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.332023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.332126 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:04.332183 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.332509 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.332580 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:40:04.332866 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:40945
--local_ip_for_outbound_sockets=127.10.207.3
--tserver_master_addrs=127.10.207.62:43323
--webserver_port=36471
--webserver_interface=127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:04.332938 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.323855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:43903)
0807 08:40:04.324092 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.324135 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.324680 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.324926 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.324929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.325558 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:04.325811 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.325815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.326581 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.326582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.327419 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.327421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.327789 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.327790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.327888 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.328429 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:04.328437 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:04.329792 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:04.331408 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.331415 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.331417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.331637 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.331844 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.331845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.331845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.331956 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.332210 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:04.332216 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:04.332329 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.332491 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.332673 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":107,"threads_started":1}
I20260807 08:40:04.333540 32654 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43323
I20260807 08:40:04.333653 32654 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:04.333858 32654 heartbeater.cc:511] Master 127.10.207.62:43323 requested a full tablet report, sending...
I20260807 08:40:04.334332 32116 ts_manager.cc:194] Registered new tserver with Master: f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135)
I20260807 08:40:04.334805 32116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:43903
W20260807 08:40:04.423714 32659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:04.423965 32659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:04.423991 32659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:04.425175 32659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:04.425225 32659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:04.425258 32659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:40:04.426859 32659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:40945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=36471
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:04.427158 32659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:04.427462 32659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:04.433228 32671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.433243 32669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:04.433619 32659 server_base.cc:1034] running on GCE node
W20260807 08:40:04.433261 32668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:04.433876 32659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:04.434099 32659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:04.435241 32659 hybrid_clock.cc:648] HybridClock initialized: now 1786092004435215 us; error 45 us; skew 500 ppm
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:04.437985 32659 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:40:04.438596 32659 webserver.cc:460] Webserver started at http://127.10.207.3:36471/ using document root <none> and password file <none>
I20260807 08:40:04.438875 32659 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:04.438977 32659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:04.440519 32659 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:40:04.441254 32682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.441525 32659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:04.441646 32659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "72d05aaf06b94642b5a2bdc4088fd078"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "62f3539d6aa12f36107e67f6411f9a93"
server_key_iv: "74c876f8ca7a41aec6800c580e24a501"
server_key_version: "encryptionkey@0"
I20260807 08:40:04.442065 32659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:04.455087 32659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:04.457541 32659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:04.457695 32659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:04.457970 32659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:04.458316 32659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:04.458369 32659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.458412 32659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:04.458441 32659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.465474 32659 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:40945
I20260807 08:40:04.465579 385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:40945 every 8 connection(s)
I20260807 08:40:04.465955 32659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:40:04.470125 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 32659
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:04.477193 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.467060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:52375 (local address 127.10.207.62:43323)
0807 08:40:04.467195 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.467198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.467404 (+ 206us) server_negotiation.cc:407] Connection header received
0807 08:40:04.468012 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.468014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.468028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.468092 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:04.469259 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.469772 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.471107 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.471307 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.471385 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.473914 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:04.473915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:04.473916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:04.473923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:04.475504 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.476069 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.476070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.476070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.476146 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.476469 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.476470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.476470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.476587 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:04.476646 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.476987 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.477072 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:40:04.477406 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.467014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:52375)
0807 08:40:04.467260 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.467310 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.467894 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.468164 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.468168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.468816 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:04.469167 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.469173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.469910 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.469911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.471006 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.471009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.471392 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.471393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.471539 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.472069 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:04.472077 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:04.473792 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:04.475602 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.475608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.475610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.475971 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.476226 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:04.476227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:04.476227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:04.476386 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:04.476668 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:04.476675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:04.476832 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.476965 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.477136 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":113,"threads_started":1}
I20260807 08:40:04.478060 386 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43323
I20260807 08:40:04.478184 386 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:04.478410 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.470600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.10.207.62:43323)
0807 08:40:04.470928 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.470930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.470937 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:40:04.470961 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.470962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.470971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.471055 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:04.471705 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.472241 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.473265 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.473530 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.473605 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.475147 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:04.475149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:04.475149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:04.475155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:04.477071 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.477478 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.477480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.477480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.477564 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:04.477871 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:04.477872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:04.477872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:04.477976 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:04.478181 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.478220 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.478307 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":69,"threads_started":1}
I20260807 08:40:04.478408 386 heartbeater.cc:511] Master 127.10.207.62:43323 requested a full tablet report, sending...
I20260807 08:40:04.478950 32116 ts_manager.cc:194] Registered new tserver with Master: 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
I20260807 08:40:04.479346 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:40:04.479425 32116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:52375
I20260807 08:40:04.482280 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.479765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37480 (local address 127.10.207.62:43323)
0807 08:40:04.479800 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.479802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.479935 (+ 133us) server_negotiation.cc:407] Connection header received
0807 08:40:04.480022 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.480023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.480059 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.480147 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:40:04.480254 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.480720 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.481473 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.481700 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.481782 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.481949 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:40:04.482058 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:40:04.482155 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.482175 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.482200 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:40:04.482676 32116 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:40:04.484807 32116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:40:04.487660 32311 tablet_service.cc:1467] Processing CreateTablet for tablet 0c72539569fc424cb6dfecccec68df5c (DEFAULT_TABLE table=test-table [id=4b52098365564936950edd14a4168336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:04.487913 32311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c72539569fc424cb6dfecccec68df5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:04.489357 32248 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap starting.
I20260807 08:40:04.489764 32248 tablet_bootstrap.cc:654] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:04.490352 32248 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: No bootstrap required, opened a new log
I20260807 08:40:04.490419 32248 ts_tablet_manager.cc:1397] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:04.490836 32248 raft_consensus.cc:348] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.490931 32248 raft_consensus.cc:374] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:04.490984 32248 raft_consensus.cc:729] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81af2c9e0e2546f09ffaba8ecdc31161, State: Initialized, Role: FOLLOWER
I20260807 08:40:04.491032 32248 consensus_queue.cc:260] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.491148 32248 ts_tablet_manager.cc:1428] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:04.491940 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.487089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41135 (local address 127.0.0.1:35202)
0807 08:40:04.487141 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.487152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.487191 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.487960 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.487961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.487964 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:04.488024 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.488026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.489547 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.489548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.490794 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.490796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.491669 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.491670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.491745 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.491767 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.491798 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.491862 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":79}
I20260807 08:40:04.492056 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.487162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:40945 (local address 127.0.0.1:59872)
0807 08:40:04.487324 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.487335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.487389 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.488109 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.488110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.488113 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:04.488164 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.488166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.489474 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.489475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.490737 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.490740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.491856 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.491857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.491912 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.491916 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.491943 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.491982 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
I20260807 08:40:04.492380 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.487782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.10.207.3:40945)
0807 08:40:04.487953 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.487956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.487965 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:40:04.487997 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.487998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.488013 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.488098 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:04.488239 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.489365 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.490841 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.491760 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.492161 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.492215 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.492252 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.492300 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260807 08:40:04.492487 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.487541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.10.207.2:41135)
0807 08:40:04.487745 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.487751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.487764 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:40:04.487809 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.487811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.487841 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.487952 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:04.488260 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.489336 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.490894 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.491565 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.492122 (+ 557us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.492187 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.492302 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.492340 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":132,"threads_started":1}
I20260807 08:40:04.493022 32759 tablet_service.cc:1467] Processing CreateTablet for tablet 0c72539569fc424cb6dfecccec68df5c (DEFAULT_TABLE table=test-table [id=4b52098365564936950edd14a4168336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:04.493189 32541 tablet_service.cc:1467] Processing CreateTablet for tablet 0c72539569fc424cb6dfecccec68df5c (DEFAULT_TABLE table=test-table [id=4b52098365564936950edd14a4168336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:04.493400 32759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c72539569fc424cb6dfecccec68df5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:04.493510 32541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c72539569fc424cb6dfecccec68df5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:04.497008 402 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Bootstrap starting.
I20260807 08:40:04.497833 402 tablet_bootstrap.cc:654] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:04.498471 402 log.cc:824] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:04.498733 403 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Bootstrap starting.
I20260807 08:40:04.499409 402 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: No bootstrap required, opened a new log
I20260807 08:40:04.499416 403 tablet_bootstrap.cc:654] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:04.499502 402 ts_tablet_manager.cc:1397] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:40:04.499969 403 log.cc:824] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:04.500929 403 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: No bootstrap required, opened a new log
I20260807 08:40:04.501035 403 ts_tablet_manager.cc:1397] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:40:04.501508 402 raft_consensus.cc:348] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.501655 402 raft_consensus.cc:374] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:04.501679 402 raft_consensus.cc:729] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72d05aaf06b94642b5a2bdc4088fd078, State: Initialized, Role: FOLLOWER
I20260807 08:40:04.501829 402 consensus_queue.cc:260] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.502125 402 ts_tablet_manager.cc:1428] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:40:04.502169 386 heartbeater.cc:503] Master 127.10.207.62:43323 was elected leader, sending a full tablet report...
I20260807 08:40:04.502961 403 raft_consensus.cc:348] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.503151 403 raft_consensus.cc:374] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:04.503190 403 raft_consensus.cc:729] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f938ccc0935b495d900cd3e2a481c903, State: Initialized, Role: FOLLOWER
I20260807 08:40:04.503317 403 consensus_queue.cc:260] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.503641 403 ts_tablet_manager.cc:1428] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:40:04.503681 32654 heartbeater.cc:503] Master 127.10.207.62:43323 was elected leader, sending a full tablet report...
W20260807 08:40:04.573678 32655 tablet.cc:2369] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:40:04.644213 32426 tablet.cc:2369] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:40:04.662009 32432 raft_consensus.cc:482] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:04.662242 32432 raft_consensus.cc:504] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.662670 32432 leader_election.cc:288] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
I20260807 08:40:04.667287 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.663040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:42213 (local address 127.10.207.3:40945)
0807 08:40:04.663092 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.663094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.663110 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:40:04.663279 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.663280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.663290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.663387 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:04.663663 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.664453 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.666130 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.666813 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.666907 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.666918 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.667155 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.667204 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:40:04.667286 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.663025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:32855 (local address 127.10.207.2:41135)
0807 08:40:04.663087 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.663090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.663110 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:40:04.663273 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.663274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.663289 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.663387 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:04.663647 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.664469 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.666130 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.666813 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.666907 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.666918 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.667154 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.667204 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:40:04.667330 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.662908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41135 (local address 127.10.207.1:32855)
0807 08:40:04.663053 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.663066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.663145 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.663414 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.663415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.663418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:04.663559 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.663560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.664580 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.664581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.665966 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.665969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.666927 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.666928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.667026 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.667052 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.667123 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.667189 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":142,"thread_start_us":67,"threads_started":1}
I20260807 08:40:04.667346 412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.662908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:40945 (local address 127.10.207.1:42213)
0807 08:40:04.663053 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.663066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:04.663145 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:04.663414 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:04.663415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:04.663418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:04.663546 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.663548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.664579 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.664581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.665966 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:04.665969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.666924 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.666925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:04.667023 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.667054 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:40:04.667124 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:40:04.667188 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":105,"thread_start_us":72,"threads_started":1}
I20260807 08:40:04.667816 32575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72539569fc424cb6dfecccec68df5c" candidate_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f938ccc0935b495d900cd3e2a481c903" is_pre_election: true
I20260807 08:40:04.667903 320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72539569fc424cb6dfecccec68df5c" candidate_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72d05aaf06b94642b5a2bdc4088fd078" is_pre_election: true
I20260807 08:40:04.668032 32575 raft_consensus.cc:2454] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81af2c9e0e2546f09ffaba8ecdc31161 in term 0.
I20260807 08:40:04.668056 320 raft_consensus.cc:2454] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81af2c9e0e2546f09ffaba8ecdc31161 in term 0.
I20260807 08:40:04.668377 32232 leader_election.cc:302] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d05aaf06b94642b5a2bdc4088fd078, 81af2c9e0e2546f09ffaba8ecdc31161; no voters:
I20260807 08:40:04.668560 32432 raft_consensus.cc:2781] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:40:04.668622 32432 raft_consensus.cc:482] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:40:04.668695 32432 raft_consensus.cc:3037] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:04.669485 32432 raft_consensus.cc:504] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.669646 32432 leader_election.cc:288] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 1 election: Requested vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
I20260807 08:40:04.669840 32575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72539569fc424cb6dfecccec68df5c" candidate_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f938ccc0935b495d900cd3e2a481c903"
I20260807 08:40:04.669847 320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c72539569fc424cb6dfecccec68df5c" candidate_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72d05aaf06b94642b5a2bdc4088fd078"
I20260807 08:40:04.669926 320 raft_consensus.cc:3037] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:04.669926 32575 raft_consensus.cc:3037] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:04.670836 320 raft_consensus.cc:2454] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81af2c9e0e2546f09ffaba8ecdc31161 in term 1.
I20260807 08:40:04.670836 32575 raft_consensus.cc:2454] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81af2c9e0e2546f09ffaba8ecdc31161 in term 1.
I20260807 08:40:04.671134 32232 leader_election.cc:302] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d05aaf06b94642b5a2bdc4088fd078, 81af2c9e0e2546f09ffaba8ecdc31161; no voters:
I20260807 08:40:04.671305 32432 raft_consensus.cc:2781] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:40:04.671471 32432 raft_consensus.cc:686] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 LEADER]: Becoming Leader. State: Replica: 81af2c9e0e2546f09ffaba8ecdc31161, State: Running, Role: LEADER
I20260807 08:40:04.671571 32432 consensus_queue.cc:237] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:04.672686 32116 catalog_manager.cc:5495] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1). New cstate: current_term: 1 leader_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } health_report { overall_health: HEALTHY } } }
I20260807 08:40:04.679050 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.676436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40318 (local address 127.10.207.1:36729)
0807 08:40:04.676576 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:04.676578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:04.676594 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:40:04.676627 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:04.676628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:04.676648 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:04.676736 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:40:04.676852 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.677361 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.678172 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:04.678386 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:04.678455 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:04.678704 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:40:04.678819 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:40:04.678892 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:04.678964 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:40:04.678987 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260807 08:40:04.680688 32575 raft_consensus.cc:1261] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Refusing update from remote peer 81af2c9e0e2546f09ffaba8ecdc31161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:04.680738 320 raft_consensus.cc:1261] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 FOLLOWER]: Refusing update from remote peer 81af2c9e0e2546f09ffaba8ecdc31161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:04.681281 416 consensus_queue.cc:1035] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:04.681440 32432 consensus_queue.cc:1035] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:04.684449 423 mvcc.cc:205] Tried to move back new op lower bound from 7315832851169280000 to 7315832851135680512. Current Snapshot: MvccSnapshot[applied={T|T < 7315832851169280000}]
I20260807 08:40:04.684391 432 mvcc.cc:205] Tried to move back new op lower bound from 7315832851169280000 to 7315832851135680512. Current Snapshot: MvccSnapshot[applied={T|T < 7315832851169280000}]
I20260807 08:40:04.686743 32115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:04.686844 32115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:04.687614 32115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Sending DeleteTablet for 3 replicas of tablet 0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:04.687951 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 32185
I20260807 08:40:04.688571 32759 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c72539569fc424cb6dfecccec68df5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:04 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:59872
I20260807 08:40:04.688814 446 tablet_replica.cc:330] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: stopping tablet replica
I20260807 08:40:04.688946 446 raft_consensus.cc:2229] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:04.689102 446 raft_consensus.cc:2258] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:04.689678 446 ts_tablet_manager.cc:1905] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:04.691968 446 ts_tablet_manager.cc:1918] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:04.692077 446 log.cc:1196] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:04.692498 446 ts_tablet_manager.cc:1939] T 0c72539569fc424cb6dfecccec68df5c P 72d05aaf06b94642b5a2bdc4088fd078: Deleting consensus metadata
I20260807 08:40:04.693034 32094 catalog_manager.cc:4841] TS 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945): tablet 0c72539569fc424cb6dfecccec68df5c (table test-table [id=4b52098365564936950edd14a4168336]) successfully deleted
I20260807 08:40:04.697988 32541 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c72539569fc424cb6dfecccec68df5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:04 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:35202
I20260807 08:40:04.699049 447 tablet_replica.cc:330] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: stopping tablet replica
I20260807 08:40:04.699187 447 raft_consensus.cc:2229] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:04.699306 447 raft_consensus.cc:2258] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:04.699731 447 ts_tablet_manager.cc:1905] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:04.700987 447 ts_tablet_manager.cc:1918] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:04.701051 447 log.cc:1196] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:04.701324 447 ts_tablet_manager.cc:1939] T 0c72539569fc424cb6dfecccec68df5c P f938ccc0935b495d900cd3e2a481c903: Deleting consensus metadata
I20260807 08:40:04.701740 32098 catalog_manager.cc:4841] TS f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135): tablet 0c72539569fc424cb6dfecccec68df5c (table test-table [id=4b52098365564936950edd14a4168336]) successfully deleted
W20260807 08:40:04.704723 32094 connection.cc:535] client connection to 127.10.207.1:36729 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:40:04.704842 32094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:40:04.705430 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 32433
I20260807 08:40:04.705461 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:04.705269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36729 (local address 127.0.0.1:40332)
0807 08:40:04.705360 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:04.705417 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 08:40:04.717432 32094 catalog_manager.cc:4584] TS 81af2c9e0e2546f09ffaba8ecdc31161 (127.10.207.1:36729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c72539569fc424cb6dfecccec68df5c: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36729: connect: Connection refused (error 111)
I20260807 08:40:04.722216 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 32659
I20260807 08:40:04.732244 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 32069
I20260807 08:40:04.747546 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43323
--webserver_interface=127.10.207.62
--webserver_port=45785
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:04.837812 449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:04.838053 449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:04.838090 449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:04.839226 449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:40:04.839298 449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:04.839322 449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:04.839344 449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:40:04.839375 449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:40:04.840857 449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=45785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:04.841138 449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:04.841377 449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:04.846611 464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:04.846743 449 server_base.cc:1034] running on GCE node
W20260807 08:40:04.846604 460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.846604 461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:04.847164 449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:04.847508 449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:04.848694 449 hybrid_clock.cc:648] HybridClock initialized: now 1786092004848672 us; error 48 us; skew 500 ppm
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:04.851894 449 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:40:04.852434 449 webserver.cc:460] Webserver started at http://127.10.207.62:45785/ using document root <none> and password file <none>
I20260807 08:40:04.852690 449 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:04.852764 449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:04.854187 449 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:40:04.855098 472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:04.855372 449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:04.855491 449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "8f439a83c322457d968c5d85a6cd21fc"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "ceeebebd5945ceea8b1a7ef5dc0b9ff9"
server_key_iv: "b32e9411ba839c5e59a9b947563578a3"
server_key_version: "encryptionkey@0"
I20260807 08:40:04.855829 449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:04.865646 449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:04.867781 449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:04.867950 449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:04.873107 449 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43323
I20260807 08:40:04.873198 554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43323 every 8 connection(s)
I20260807 08:40:04.873611 449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:40:04.874214 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 449
I20260807 08:40:04.874686 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:36729
--local_ip_for_outbound_sockets=127.10.207.1
--tserver_master_addrs=127.10.207.62:43323
--webserver_port=44321
--webserver_interface=127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:04.876451 555 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:40:04.877528 555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Bootstrap starting.
I20260807 08:40:04.890018 555 log.cc:824] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:04.892792 555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:40:04.893203 555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc: Bootstrap complete.
I20260807 08:40:04.896095 555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.896385 555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f439a83c322457d968c5d85a6cd21fc, State: Initialized, Role: FOLLOWER
I20260807 08:40:04.896595 555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.896673 555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:40:04.896708 555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:40:04.896788 555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:40:04.898196 555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.898353 555 leader_election.cc:302] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f439a83c322457d968c5d85a6cd21fc; no voters:
I20260807 08:40:04.898600 555 leader_election.cc:288] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:40:04.898651 559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:40:04.898834 559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [term 3 LEADER]: Becoming Leader. State: Replica: 8f439a83c322457d968c5d85a6cd21fc, State: Running, Role: LEADER
I20260807 08:40:04.898950 555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:40:04.898937 559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } }
I20260807 08:40:04.899304 559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f439a83c322457d968c5d85a6cd21fc. Latest consensus state: current_term: 3 leader_uuid: "8f439a83c322457d968c5d85a6cd21fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } } }
I20260807 08:40:04.899399 559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:04.899308 561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8f439a83c322457d968c5d85a6cd21fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f439a83c322457d968c5d85a6cd21fc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43323 } } }
I20260807 08:40:04.899813 561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f439a83c322457d968c5d85a6cd21fc [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:04.899652 566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:40:04.900915 566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:40:04.901113 566 catalog_manager.cc:1259] Loaded cluster ID: 039b3f3ad2304300a0ec3e09864f3a70
I20260807 08:40:04.901418 566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 08:40:04.901929 566 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6fb9ac79ad google::LogMessage::Fail() at ??:0
@ 0x7f6fb9aca88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6fb9ac74de google::LogMessage::Flush() at ??:0
@ 0x7f6fb9ac8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6fc19e80f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6fc19f18b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6fbb9950d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6fbb98a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6fbc7ee6db start_thread at ??:0
@ 0x7f6fb85c561f clone at ??:0
W20260807 08:40:04.983892 557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:04.984156 557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:04.984198 557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:04.985400 557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:04.985471 557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:04.985518 557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:40:04.987146 557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:36729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=44321
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:04.987434 557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:04.987703 557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:04.993669 615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.993677 613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:04.993677 618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:04.994406 557 server_base.cc:1034] running on GCE node
I20260807 08:40:04.994678 557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:04.994968 557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:04.996153 557 hybrid_clock.cc:648] HybridClock initialized: now 1786092004996141 us; error 46 us; skew 500 ppm
Aug 07 08:40:04 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:04.999518 557 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:40:05.000157 557 webserver.cc:460] Webserver started at http://127.10.207.1:44321/ using document root <none> and password file <none>
I20260807 08:40:05.000445 557 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:05.000527 557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:05.002449 557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:40:05.003353 628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:05.003598 557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:05.003719 557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "81af2c9e0e2546f09ffaba8ecdc31161"
format_stamp: "Formatted at 2026-08-07 08:40:02 on dist-test-slave-h6n0"
server_key: "a33aeb9852738bb783de281bb2288c07"
server_key_iv: "de14a13592cefc4309ed102eb6fb4795"
server_key_version: "encryptionkey@0"
I20260807 08:40:05.004226 557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:05.034330 557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:05.036891 557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:05.037060 557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:05.037360 557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:05.037926 644 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:40:05.039870 557 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:40:05.039943 557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:40:05.039980 557 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:40:05.041025 557 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:40:05.041076 557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:05.041252 644 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap starting.
I20260807 08:40:05.049628 557 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:36729
I20260807 08:40:05.049861 814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:36729 every 8 connection(s)
I20260807 08:40:05.050117 557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:40:05.052429 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 557
I20260807 08:40:05.052729 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:41135
--local_ip_for_outbound_sockets=127.10.207.2
--tserver_master_addrs=127.10.207.62:43323
--webserver_port=44669
--webserver_interface=127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:05.055055 644 log.cc:824] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:05.058115 644 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:40:05.058481 644 tablet_bootstrap.cc:492] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Bootstrap complete.
I20260807 08:40:05.059921 644 ts_tablet_manager.cc:1397] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.009s
I20260807 08:40:05.061903 644 raft_consensus.cc:348] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:05.062343 644 raft_consensus.cc:729] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81af2c9e0e2546f09ffaba8ecdc31161, State: Initialized, Role: FOLLOWER
I20260807 08:40:05.062594 644 consensus_queue.cc:260] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:05.062981 644 ts_tablet_manager.cc:1428] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
W20260807 08:40:05.113401 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:40:05.051052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:45495)
0807 08:40:05.051306 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.051367 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:05.052395 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:05.113114 (+ 60719us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":121,"threads_started":1}
W20260807 08:40:05.113704 815 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:43323 (0 consecutive failures): Network error: Failed to ping master at 127.10.207.62:43323: Client connection negotiation failed: client connection to 127.10.207.62:43323: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:40:05.114091 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.113961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:33571)
0807 08:40:05.114021 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.114070 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:40:05.114439 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.114334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:54053)
0807 08:40:05.114383 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.114409 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 08:40:05.114615 815 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:40:05.155138 820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:05.155393 820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:05.155422 820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:05.156672 820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:05.156716 820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:05.156746 820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:40:05.158325 820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:41135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=44669
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:05.158645 820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:05.158918 820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:05.164587 833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:05.164587 839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:05.164642 836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:05.165015 820 server_base.cc:1034] running on GCE node
I20260807 08:40:05.165282 820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:05.165555 820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:05.166733 820 hybrid_clock.cc:648] HybridClock initialized: now 1786092005166717 us; error 39 us; skew 500 ppm
Aug 07 08:40:05 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092005, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:05.169615 820 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:40:05.170135 820 webserver.cc:460] Webserver started at http://127.10.207.2:44669/ using document root <none> and password file <none>
I20260807 08:40:05.170399 820 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:05.170454 820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:05.172194 820 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:05.173100 852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:05.173302 820 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:05.173403 820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "f938ccc0935b495d900cd3e2a481c903"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "e690ac2b44b5a9691720a4732d3b8e1d"
server_key_iv: "42ea579cc803740e1f302a4c72fbd1da"
server_key_version: "encryptionkey@0"
I20260807 08:40:05.173861 820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:05.181495 820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:05.183835 820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:05.183954 820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:05.184152 820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:05.184441 820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:05.184484 820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:05.184505 820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:05.184518 820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:05.191975 820 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:41135
I20260807 08:40:05.192075 1027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:41135 every 8 connection(s)
I20260807 08:40:05.192433 820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:40:05.193085 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 820
I20260807 08:40:05.193379 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:40945
--local_ip_for_outbound_sockets=127.10.207.3
--tserver_master_addrs=127.10.207.62:43323
--webserver_port=36471
--webserver_interface=127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:05.193813 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.193271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:50761)
0807 08:40:05.193590 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.193664 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
W20260807 08:40:05.194416 1028 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:43323 (0 consecutive failures): Network error: Failed to ping master at 127.10.207.62:43323: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
I20260807 08:40:05.194803 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.194697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:43069)
0807 08:40:05.194752 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.194779 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:40:05.195199 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.195081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:58501)
0807 08:40:05.195153 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.195178 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 08:40:05.195358 1028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:40:05.287384 1035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:05.287622 1035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:05.287648 1035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:05.288841 1035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:05.288885 1035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:05.288913 1035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:40:05.290506 1035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:40945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=36471
--tserver_master_addrs=127.10.207.62:43323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:05.290851 1035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:05.291157 1035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:05.296917 1045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:05.296917 1044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:05.297029 1047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:05.297188 1035 server_base.cc:1034] running on GCE node
I20260807 08:40:05.297432 1035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:05.297771 1035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:05.298957 1035 hybrid_clock.cc:648] HybridClock initialized: now 1786092005298943 us; error 44 us; skew 500 ppm
Aug 07 08:40:05 dist-test-slave-h6n0 krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092005, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:05.301887 1035 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:40:05.302450 1035 webserver.cc:460] Webserver started at http://127.10.207.3:36471/ using document root <none> and password file <none>
I20260807 08:40:05.302738 1035 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:05.302815 1035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:05.304502 1035 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:40:05.305322 1056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:05.305596 1035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:05.305719 1035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "72d05aaf06b94642b5a2bdc4088fd078"
format_stamp: "Formatted at 2026-08-07 08:40:03 on dist-test-slave-h6n0"
server_key: "62f3539d6aa12f36107e67f6411f9a93"
server_key_iv: "74c876f8ca7a41aec6800c580e24a501"
server_key_version: "encryptionkey@0"
I20260807 08:40:05.306149 1035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:05.319491 1035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:05.322209 1035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:05.322386 1035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:05.322674 1035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:05.323024 1035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:05.323078 1035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:05.323132 1035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:05.323160 1035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:05.331058 1035 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:40945
I20260807 08:40:05.331236 1221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:40945 every 8 connection(s)
I20260807 08:40:05.331571 1035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:40:05.333012 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.332474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:47279)
0807 08:40:05.332697 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.332804 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":85,"threads_started":1}
W20260807 08:40:05.333536 1223 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:43323 (0 consecutive failures): Network error: Failed to ping master at 127.10.207.62:43323: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
I20260807 08:40:05.333850 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.333736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:43415)
0807 08:40:05.333792 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.333825 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:40:05.334223 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:05.334132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:34427)
0807 08:40:05.334184 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:05.334208 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:40:05.334384 1223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:40:05.340884 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 1035
I20260807 08:40:06.115309 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:06.115024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:58971)
0807 08:40:06.115191 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:06.115234 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260807 08:40:06.196019 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:06.195724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:36127)
0807 08:40:06.195868 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:06.195921 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260807 08:40:06.335090 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:06.334783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:52199)
0807 08:40:06.334951 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:06.335017 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260807 08:40:06.862553 1236 raft_consensus.cc:482] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:06.862715 1236 raft_consensus.cc:504] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:06.863085 1236 leader_election.cc:288] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
Aug 07 08:40:06 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.2@KRBTEST.COM
Aug 07 08:40:06 dist-test-slave-h6n0 krb5kdc[31178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092004, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.3@KRBTEST.COM
I20260807 08:40:06.881096 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:06.863536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:47357 (local address 127.10.207.3:40945)
0807 08:40:06.863673 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:06.863680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:06.863699 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:40:06.864256 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:06.864259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:06.864316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:06.864502 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:06.865815 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.867261 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.869021 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.869644 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.869765 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:06.877336 (+ 7571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:06.877340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:06.877340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:06.877351 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:06.879238 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:06.879795 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:06.879798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:06.879799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:06.879896 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:06.880179 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:06.880179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:06.880180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:06.880419 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:06.880510 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:06.880827 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:40:06.880963 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":53,"threads_started":1}
I20260807 08:40:06.881099 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:06.863301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:40945 (local address 127.10.207.1:47357)
0807 08:40:06.863497 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:06.863507 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:06.863585 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:06.864634 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:06.864636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:06.865286 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:06.865657 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:06.865667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.867407 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.867407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:06.868900 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:06.868904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.869766 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.869766 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:06.869947 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:06.874577 (+ 4630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:06.874595 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:06.877220 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:06.879356 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:06.879357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:06.879357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:06.879709 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:06.879977 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:06.879978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:06.879978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:06.880102 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:06.880512 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:06.880521 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:06.880658 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 08:40:06.880772 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 08:40:06.881003 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":99,"thread_start_us":89,"threads_started":1}
I20260807 08:40:06.881139 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:06.863426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:34071 (local address 127.10.207.2:41135)
0807 08:40:06.863625 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:06.863638 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:06.863666 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:40:06.864256 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:06.864259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:06.864316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:06.864502 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:06.865815 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.867256 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.869012 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.869644 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.869765 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:06.877044 (+ 7279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:06.877050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:06.877050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:06.877063 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:06.879070 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:06.879817 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:06.879821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:06.879821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:06.879929 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:06.880185 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:06.880186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:06.880186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:06.880464 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:06.880565 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:06.880839 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 08:40:06.880987 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
I20260807 08:40:06.881223 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:06.863301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41135 (local address 127.10.207.1:34071)
0807 08:40:06.863493 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:06.863507 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:06.863585 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:06.864632 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:06.864636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:06.865286 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:06.865657 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:06.865667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.867405 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.867406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:06.868900 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:06.868904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:06.869765 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:06.869766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:06.869947 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:06.874577 (+ 4630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:06.874590 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:06.876894 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:06.879309 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:06.879318 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:06.879321 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:06.879706 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:06.880010 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:06.880011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:06.880012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:06.880111 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:06.880542 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:06.880543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:06.880666 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 08:40:06.880774 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:40:06.881107 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":317,"thread_start_us":87,"threads_started":1}
W20260807 08:40:06.882221 636 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
W20260807 08:40:06.882794 630 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:06.882874 630 leader_election.cc:302] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81af2c9e0e2546f09ffaba8ecdc31161; no voters: 72d05aaf06b94642b5a2bdc4088fd078, f938ccc0935b495d900cd3e2a481c903
I20260807 08:40:06.883050 1236 raft_consensus.cc:2726] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:40:07.116165 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:07.115966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:53295)
0807 08:40:07.116035 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:07.116083 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 08:40:07.196929 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:07.196632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:46315)
0807 08:40:07.196811 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:07.196871 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260807 08:40:07.335994 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:07.335651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:50733)
0807 08:40:07.335857 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:07.335931 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":124,"threads_started":1}
I20260807 08:40:08.117080 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:08.116788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:42613)
0807 08:40:08.116969 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:08.117021 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260807 08:40:08.197853 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:08.197555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:58599)
0807 08:40:08.197751 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:08.197798 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260807 08:40:08.336982 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:08.336694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:38111)
0807 08:40:08.336865 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:08.336922 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
I20260807 08:40:08.630123 1256 raft_consensus.cc:482] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:08.630242 1256 raft_consensus.cc:504] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:08.630493 1256 leader_election.cc:288] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
W20260807 08:40:08.631121 636 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
W20260807 08:40:08.631237 630 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:08.631317 630 leader_election.cc:302] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81af2c9e0e2546f09ffaba8ecdc31161; no voters: 72d05aaf06b94642b5a2bdc4088fd078, f938ccc0935b495d900cd3e2a481c903
I20260807 08:40:08.631448 1256 raft_consensus.cc:2726] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:40:09.117925 1258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:09.117614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:50687)
0807 08:40:09.117768 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:09.117812 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260807 08:40:09.199057 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:09.198650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:39863)
0807 08:40:09.198843 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:09.198950 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260807 08:40:09.337821 1260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:09.337587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:38657)
0807 08:40:09.337739 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:09.337776 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260807 08:40:10.118873 1262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:10.118576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:40447)
0807 08:40:10.118763 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:10.118813 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
I20260807 08:40:10.199863 1264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:10.199577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:52555)
0807 08:40:10.199764 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:10.199805 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260807 08:40:10.338819 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:10.338466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:54939)
0807 08:40:10.338711 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:10.338760 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":163,"threads_started":1}
I20260807 08:40:10.488142 1267 raft_consensus.cc:482] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:10.488370 1267 raft_consensus.cc:504] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:10.488615 1267 leader_election.cc:288] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
W20260807 08:40:10.489056 636 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
W20260807 08:40:10.489149 630 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:10.489292 630 leader_election.cc:302] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81af2c9e0e2546f09ffaba8ecdc31161; no voters: 72d05aaf06b94642b5a2bdc4088fd078, f938ccc0935b495d900cd3e2a481c903
I20260807 08:40:10.489392 1267 raft_consensus.cc:2726] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:40:11.119722 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:11.119418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:36815)
0807 08:40:11.119616 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:11.119669 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":101,"threads_started":1}
I20260807 08:40:11.200663 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:11.200377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:53345)
0807 08:40:11.200559 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:11.200598 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
I20260807 08:40:11.339564 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:11.339309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:35419)
0807 08:40:11.339489 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:11.339528 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260807 08:40:12.120571 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:12.120260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:51909)
0807 08:40:12.120442 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:12.120504 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260807 08:40:12.201475 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:12.201175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:49765)
0807 08:40:12.201347 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:12.201401 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260807 08:40:12.340562 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:12.340250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:44031)
0807 08:40:12.340457 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:12.340510 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
I20260807 08:40:12.930043 1278 raft_consensus.cc:482] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:12.930173 1278 raft_consensus.cc:504] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:12.930395 1278 leader_election.cc:288] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
W20260807 08:40:12.930826 630 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
W20260807 08:40:12.930964 636 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:12.931015 636 leader_election.cc:302] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81af2c9e0e2546f09ffaba8ecdc31161; no voters: 72d05aaf06b94642b5a2bdc4088fd078, f938ccc0935b495d900cd3e2a481c903
I20260807 08:40:12.931108 1278 raft_consensus.cc:2726] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:40:13.121431 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:13.121133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:40833)
0807 08:40:13.121312 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:13.121365 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260807 08:40:13.202313 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:13.202030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:36031)
0807 08:40:13.202213 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:13.202256 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":108,"threads_started":1}
I20260807 08:40:13.341360 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:13.341124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:57097)
0807 08:40:13.341289 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:13.341326 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
I20260807 08:40:14.122272 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:14.121990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:44415)
0807 08:40:14.122169 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:14.122220 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":84,"threads_started":1}
I20260807 08:40:14.203210 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:14.202836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:49219)
0807 08:40:14.203029 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:14.203153 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260807 08:40:14.342207 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:14.341895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:42463)
0807 08:40:14.342083 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:14.342146 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260807 08:40:15.123163 1290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:15.122866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:51747)
0807 08:40:15.123060 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:15.123114 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":103,"threads_started":1}
I20260807 08:40:15.204005 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:15.203721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:32905)
0807 08:40:15.203897 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:15.203944 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260807 08:40:15.343101 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:15.342820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:54737)
0807 08:40:15.342979 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:15.343031 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
I20260807 08:40:15.528986 1294 raft_consensus.cc:482] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:15.529136 1294 raft_consensus.cc:504] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f938ccc0935b495d900cd3e2a481c903" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41135 } } peers { permanent_uuid: "72d05aaf06b94642b5a2bdc4088fd078" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 40945 } } peers { permanent_uuid: "81af2c9e0e2546f09ffaba8ecdc31161" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36729 } }
I20260807 08:40:15.529450 1294 leader_election.cc:288] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135), 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945)
W20260807 08:40:15.529912 636 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f938ccc0935b495d900cd3e2a481c903 (127.10.207.2:41135): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
W20260807 08:40:15.530021 630 leader_election.cc:341] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 72d05aaf06b94642b5a2bdc4088fd078 (127.10.207.3:40945): Not found: Tablet not found: 0c72539569fc424cb6dfecccec68df5c
I20260807 08:40:15.530055 636 leader_election.cc:302] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81af2c9e0e2546f09ffaba8ecdc31161; no voters: 72d05aaf06b94642b5a2bdc4088fd078, f938ccc0935b495d900cd3e2a481c903
I20260807 08:40:15.530169 1294 raft_consensus.cc:2726] T 0c72539569fc424cb6dfecccec68df5c P 81af2c9e0e2546f09ffaba8ecdc31161 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:40:16.124061 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:16.123797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:58033)
0807 08:40:16.123970 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:16.124010 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260807 08:40:16.204813 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:16.204538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:53611)
0807 08:40:16.204703 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:16.204756 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260807 08:40:16.343935 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:16.343653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:56901)
0807 08:40:16.343815 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:16.343873 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260807 08:40:17.124866 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:17.124606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:36701)
0807 08:40:17.124765 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:17.124814 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260807 08:40:17.205709 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:17.205446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:42121)
0807 08:40:17.205624 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:17.205667 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260807 08:40:17.344898 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:17.344611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:59953)
0807 08:40:17.344799 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:17.344847 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
I20260807 08:40:18.125721 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:18.125437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:39589)
0807 08:40:18.125616 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:18.125670 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260807 08:40:18.206708 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:18.206311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:46241)
0807 08:40:18.206588 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:18.206656 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":111,"threads_started":1}
I20260807 08:40:18.345806 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:18.345513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:41781)
0807 08:40:18.345691 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:18.345741 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260807 08:40:19.126703 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:19.126340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:41087)
0807 08:40:19.126517 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:19.126638 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260807 08:40:19.207513 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:19.207252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:56691)
0807 08:40:19.207405 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:19.207449 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260807 08:40:19.346729 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:19.346371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:46495)
0807 08:40:19.346619 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:19.346668 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":87,"threads_started":1}
I20260807 08:40:20.127909 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:20.127529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.1:32953)
0807 08:40:20.127796 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:20.127849 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":141,"threads_started":1}
I20260807 08:40:20.208568 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:20.208218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.2:57669)
0807 08:40:20.208447 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:20.208502 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":102,"threads_started":1}
I20260807 08:40:20.347684 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:20.347359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43323 (local address 127.10.207.3:55377)
0807 08:40:20.347550 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:20.347623 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:43323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
I20260807 08:40:20.348886 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 557
I20260807 08:40:20.370821 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 820
I20260807 08:40:20.392423 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 1035
2026-08-07T08:40:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19151 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:40:20 dist-test-slave-h6n0 krb5kdc[1322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:40:20 dist-test-slave-h6n0 krb5kdc[1322](info): set up 2 sockets
Aug 07 08:40:20 dist-test-slave-h6n0 krb5kdc[1322](info): commencing operation
krb5kdc: starting...
W20260807 08:40:21.719395 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:40:21 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:40:21.855168 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45045
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:45045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:21.948457 1338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:21.948819 1338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:21.948858 1338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:21.950747 1338 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:40:21.950804 1338 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:40:21.950829 1338 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:40:21.950852 1338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:40:21.950886 1338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:21.950915 1338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:21.950945 1338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:40:21.950968 1338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:40:21.952981 1338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:45045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:21.953871 1338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:21.954144 1338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:21.961077 1350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:21.961138 1348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:21.961217 1352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:21.961555 1338 server_base.cc:1034] running on GCE node
I20260807 08:40:21.961830 1338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:21.962113 1338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:21.963279 1338 hybrid_clock.cc:648] HybridClock initialized: now 1786092021963252 us; error 42 us; skew 500 ppm
Aug 07 08:40:21 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092021, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:21.967177 1338 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:40:21.967815 1338 webserver.cc:460] Webserver started at http://127.10.207.62:43593/ using document root <none> and password file <none>
I20260807 08:40:21.968137 1338 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:21.968212 1338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:21.968369 1338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:21.969926 1338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "1434d9844f1f4b1f893a0fffb289ae34"
format_stamp: "Formatted at 2026-08-07 08:40:21 on dist-test-slave-h6n0"
server_key: "9a683d3bc4ff79fe2fd1e55e30a84c57"
server_key_iv: "214bb7414af029ca8dda88927902f9c2"
server_key_version: "encryptionkey@0"
I20260807 08:40:21.970423 1338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "1434d9844f1f4b1f893a0fffb289ae34"
format_stamp: "Formatted at 2026-08-07 08:40:21 on dist-test-slave-h6n0"
server_key: "9a683d3bc4ff79fe2fd1e55e30a84c57"
server_key_iv: "214bb7414af029ca8dda88927902f9c2"
server_key_version: "encryptionkey@0"
I20260807 08:40:21.972573 1338 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 08:40:21.973649 1359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:21.973858 1338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:21.973996 1338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "1434d9844f1f4b1f893a0fffb289ae34"
format_stamp: "Formatted at 2026-08-07 08:40:21 on dist-test-slave-h6n0"
server_key: "9a683d3bc4ff79fe2fd1e55e30a84c57"
server_key_iv: "214bb7414af029ca8dda88927902f9c2"
server_key_version: "encryptionkey@0"
I20260807 08:40:21.974098 1338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:21.997443 1338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:22.000988 1338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:22.001194 1338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:22.007784 1338 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:45045
I20260807 08:40:22.007851 1445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:45045 every 8 connection(s)
I20260807 08:40:22.008270 1338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:40:22.009384 1446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:22.011581 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 1338
I20260807 08:40:22.011709 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:40:22.011929 11068 external_mini_cluster.cc:1428] Setting key b0421711eed553d405fbcf741a82667d
I20260807 08:40:22.012147 1446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Bootstrap starting.
I20260807 08:40:22.013267 1446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:22.013600 1446 log.cc:824] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:22.014333 1446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: No bootstrap required, opened a new log
I20260807 08:40:22.016551 1446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:22.016743 1446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:22.016793 1446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1434d9844f1f4b1f893a0fffb289ae34, State: Initialized, Role: FOLLOWER
I20260807 08:40:22.016955 1446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:22.017046 1446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:40:22.017099 1446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:40:22.017143 1446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:22.017750 1446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:22.017879 1446 leader_election.cc:302] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1434d9844f1f4b1f893a0fffb289ae34; no voters:
I20260807 08:40:22.018085 1446 leader_election.cc:288] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:40:22.018187 1451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:40:22.018406 1451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 LEADER]: Becoming Leader. State: Replica: 1434d9844f1f4b1f893a0fffb289ae34, State: Running, Role: LEADER
I20260807 08:40:22.018429 1446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:40:22.018656 1451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:22.019146 1452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1434d9844f1f4b1f893a0fffb289ae34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } } }
I20260807 08:40:22.019192 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1434d9844f1f4b1f893a0fffb289ae34. Latest consensus state: current_term: 1 leader_uuid: "1434d9844f1f4b1f893a0fffb289ae34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } } }
I20260807 08:40:22.019266 1452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:22.019274 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:22.019526 1466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:40:22.020179 1466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:40:22 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:22.022786 1466 catalog_manager.cc:1347] Generated new cluster ID: 5ddf985dde9d49d0bb1401d9c073355c
I20260807 08:40:22.022858 1466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:40:22.029640 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.012714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.10.207.62:45045)
0807 08:40:22.012974 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.012984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.013080 (+ 96us) server_negotiation.cc:407] Connection header received
0807 08:40:22.013789 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.013798 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.013856 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.014054 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:22.014945 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.015939 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.017070 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.017629 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.017762 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.021442 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:22.021446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:22.021447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:22.021458 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:22.027720 (+ 6262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.028212 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.028216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.028216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.028328 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.028582 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.028583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.028583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.028902 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:22.029045 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.029247 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.029401 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":120,"threads_started":1}
I20260807 08:40:22.036737 1466 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:40:22.037355 1466 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:40:22.051514 1466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Generated new TSK 0
I20260807 08:40:22.051740 1466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:40:22.085892 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45045
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:22.197403 1490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:22.197598 1490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:22.197628 1490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:22.198745 1490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:22.198783 1490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:22.198810 1490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:40:22.200165 1490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:22.200356 1490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:22.200543 1490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:22.206732 1497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:22.206753 1496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:22.206755 1501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:22.206924 1490 server_base.cc:1034] running on GCE node
I20260807 08:40:22.207355 1490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:22.207585 1490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:22.208801 1490 hybrid_clock.cc:648] HybridClock initialized: now 1786092022208775 us; error 46 us; skew 500 ppm
Aug 07 08:40:22 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092022, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:22.211740 1490 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:40:22.212210 1490 webserver.cc:460] Webserver started at http://127.10.207.1:45111/ using document root <none> and password file <none>
I20260807 08:40:22.212461 1490 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:22.212512 1490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:22.212666 1490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:22.213730 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "e9371bedbaa044a99fbd92052dd6630b"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d47ededd34ecb05435b18a72afba18b6"
server_key_iv: "e0c84ed29890f66186480a16c73097d9"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.214098 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "e9371bedbaa044a99fbd92052dd6630b"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d47ededd34ecb05435b18a72afba18b6"
server_key_iv: "e0c84ed29890f66186480a16c73097d9"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.215647 1490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:40:22.216559 1512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.216823 1490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:22.216912 1490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "e9371bedbaa044a99fbd92052dd6630b"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d47ededd34ecb05435b18a72afba18b6"
server_key_iv: "e0c84ed29890f66186480a16c73097d9"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.217015 1490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:22.242261 1490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:22.244650 1490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:22.244922 1490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:22.245213 1490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:22.245556 1490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:22.245595 1490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.245618 1490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:22.245632 1490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.253260 1490 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:39873
I20260807 08:40:22.253327 1689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:39873 every 8 connection(s)
I20260807 08:40:22.253633 1490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:40:22.254302 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 1490
I20260807 08:40:22.254439 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:40:22.254714 11068 external_mini_cluster.cc:1428] Setting key fe54f4f71ec69a7e1f9ba0588590329c
Aug 07 08:40:22 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092022, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:22.264146 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.254410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:35635 (local address 127.10.207.62:45045)
0807 08:40:22.254485 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.254489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.254707 (+ 218us) server_negotiation.cc:407] Connection header received
0807 08:40:22.255351 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.255353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.255366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.255448 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:22.256406 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.256942 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.257924 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.258149 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.258228 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.260799 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:22.260800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:22.260800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:22.260807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:22.262269 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.262922 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.262924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.262924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.262993 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.263362 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.263362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.263363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.263477 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:22.263537 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.263997 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.264079 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:40:22.264485 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.254393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:35635)
0807 08:40:22.254576 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.254618 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.255236 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.255536 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.255540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.256104 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:22.256294 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.256298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.257049 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.257051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.257832 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.257833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.258237 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.258239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.258357 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.258926 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:22.258934 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:22.260666 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:22.262372 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:22.262381 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:22.262384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:22.262810 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:22.263095 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:22.263096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:22.263096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:22.263265 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:22.263561 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:22.263570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:22.263814 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.263955 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.264178 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260807 08:40:22.265211 1690 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45045
I20260807 08:40:22.265307 1690 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
I20260807 08:40:22.265498 1690 heartbeater.cc:511] Master 127.10.207.62:45045 requested a full tablet report, sending...
I20260807 08:40:22.266232 1390 ts_manager.cc:194] Registered new tserver with Master: e9371bedbaa044a99fbd92052dd6630b (127.10.207.1:39873)
I20260807 08:40:22.267027 1390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:35635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:40:22.285960 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45045
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:22.369021 1695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:22.369205 1695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:22.369228 1695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:22.370314 1695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:22.370358 1695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:22.370383 1695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:40:22.371906 1695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:22.372138 1695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:22.372376 1695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:22.377595 1702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:22.377615 1709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:22.377595 1701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:22.377789 1695 server_base.cc:1034] running on GCE node
I20260807 08:40:22.378141 1695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:22.378398 1695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:22.379554 1695 hybrid_clock.cc:648] HybridClock initialized: now 1786092022379520 us; error 57 us; skew 500 ppm
Aug 07 08:40:22 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092022, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:22.382264 1695 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:40:22.382790 1695 webserver.cc:460] Webserver started at http://127.10.207.2:44075/ using document root <none> and password file <none>
I20260807 08:40:22.383021 1695 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:22.383091 1695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:22.383234 1695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:22.384227 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "7731efc76a9b4af6940a23e598c7a2af"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "34e1773f8ebf322e327b30edeb5427a1"
server_key_iv: "80b56acf964ae7a2bebb21e4a760b391"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.384582 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "7731efc76a9b4af6940a23e598c7a2af"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "34e1773f8ebf322e327b30edeb5427a1"
server_key_iv: "80b56acf964ae7a2bebb21e4a760b391"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.386003 1695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:40:22.386966 1721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.387225 1695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:22.387344 1695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "7731efc76a9b4af6940a23e598c7a2af"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "34e1773f8ebf322e327b30edeb5427a1"
server_key_iv: "80b56acf964ae7a2bebb21e4a760b391"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.387437 1695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:22.396251 1695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:22.398496 1695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:22.398674 1695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:22.398917 1695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:22.399228 1695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:22.399286 1695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.399320 1695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:22.399348 1695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.406208 1695 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:33335
I20260807 08:40:22.406273 1908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:33335 every 8 connection(s)
I20260807 08:40:22.406615 1695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:40:22.411818 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 1695
I20260807 08:40:22.411952 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:40:22.412209 11068 external_mini_cluster.cc:1428] Setting key 1ecb5d15a495180418511ac7c17e0d8b
Aug 07 08:40:22 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092022, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:22.417689 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.407473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:36261 (local address 127.10.207.62:45045)
0807 08:40:22.407567 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.407570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.407800 (+ 230us) server_negotiation.cc:407] Connection header received
0807 08:40:22.408456 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.408458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.408472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.408532 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:22.409628 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.410187 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.411319 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.411519 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.411606 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.414265 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:22.414267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:22.414267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:22.414276 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:22.416080 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.416558 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.416560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.416560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.416628 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.416908 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.416909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.416909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.417047 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:22.417148 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.417485 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.417582 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:40:22.418694 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.407365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:36261)
0807 08:40:22.407610 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.407685 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.408335 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.408614 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.408627 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.409289 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:22.409516 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.409520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.410294 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.410295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.411216 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.411219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.411619 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.411626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.411762 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.412424 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:22.412439 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:22.414148 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:22.416201 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:22.416211 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:22.416213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:22.416471 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:22.416702 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:22.416703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:22.416703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:22.416833 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:22.417142 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:22.417150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:22.417318 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.417450 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.417645 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":105,"threads_started":1}
I20260807 08:40:22.419430 1909 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45045
I20260807 08:40:22.419559 1909 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:22.419831 1909 heartbeater.cc:511] Master 127.10.207.62:45045 requested a full tablet report, sending...
I20260807 08:40:22.420521 1390 ts_manager.cc:194] Registered new tserver with Master: 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
I20260807 08:40:22.421017 1390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:36261
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:40:22.445508 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45045
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:22.539660 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:22.539888 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:22.539915 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:22.541080 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:22.541128 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:22.541157 1918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:40:22.542717 1918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:22.542984 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:22.543252 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:22.549468 1927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:22.549500 1928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:22.549466 1930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:22.549732 1918 server_base.cc:1034] running on GCE node
I20260807 08:40:22.550024 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:22.550366 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:22.551561 1918 hybrid_clock.cc:648] HybridClock initialized: now 1786092022551541 us; error 66 us; skew 500 ppm
Aug 07 08:40:22 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092022, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:22.554373 1918 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:40:22.554911 1918 webserver.cc:460] Webserver started at http://127.10.207.3:35157/ using document root <none> and password file <none>
I20260807 08:40:22.555160 1918 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:22.555230 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:22.555378 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:40:22.556524 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "e507b3fc1d7146e49f03c45f7c6976dc"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d6ca6beeabb254321550eeb63a9f126b"
server_key_iv: "1b76a43e440e947e315591c72c303799"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.556970 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "e507b3fc1d7146e49f03c45f7c6976dc"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d6ca6beeabb254321550eeb63a9f126b"
server_key_iv: "1b76a43e440e947e315591c72c303799"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.558663 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:40:22.559721 1941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.560062 1918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:22.560196 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "e507b3fc1d7146e49f03c45f7c6976dc"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d6ca6beeabb254321550eeb63a9f126b"
server_key_iv: "1b76a43e440e947e315591c72c303799"
server_key_version: "encryptionkey@0"
I20260807 08:40:22.560276 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:22.582356 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:22.585314 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:22.585490 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:22.585749 1918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:22.586081 1918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:22.586138 1918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.586174 1918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:22.586215 1918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:22.593046 1918 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34759
I20260807 08:40:22.593160 2118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34759 every 8 connection(s)
I20260807 08:40:22.593488 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:40:22.594599 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 1918
I20260807 08:40:22.594686 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:40:22.594875 11068 external_mini_cluster.cc:1428] Setting key fce041c481987e183f7ac49c10b53841
Aug 07 08:40:22 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092022, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:22.603869 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.594353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:49423 (local address 127.10.207.62:45045)
0807 08:40:22.594456 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.594460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.594700 (+ 240us) server_negotiation.cc:407] Connection header received
0807 08:40:22.595430 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.595431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.595447 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.595497 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:22.596659 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.597238 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.598320 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.598588 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.598688 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.600994 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:22.600995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:22.600996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:22.601004 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:22.602302 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.602807 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.602808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.602808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.602895 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.603174 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.603175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.603175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.603301 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:22.603378 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.603672 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.603762 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 08:40:22.604084 2121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.594250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:49423)
0807 08:40:22.594494 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.594572 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.595297 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.595602 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.595607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.596312 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:22.596552 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.596557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.597346 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.597347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.598225 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.598227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.598701 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.598703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.598857 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.599377 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:22.599386 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:22.600891 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:22.602394 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:22.602402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:22.602405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:22.602712 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:22.602966 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:22.602967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:22.602968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:22.603091 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:22.603377 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:22.603386 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:22.603533 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.603649 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.603825 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":101,"threads_started":1}
I20260807 08:40:22.604736 2119 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45045
I20260807 08:40:22.604836 2119 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:22.605067 2119 heartbeater.cc:511] Master 127.10.207.62:45045 requested a full tablet report, sending...
I20260807 08:40:22.605587 1390 ts_manager.cc:194] Registered new tserver with Master: e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759)
I20260807 08:40:22.606065 1390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:49423
I20260807 08:40:22.607199 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:40:22.615660 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.608868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.10.207.62:45045)
0807 08:40:22.608968 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.608970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.609011 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:40:22.609080 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.609081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.609090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.609145 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:22.609776 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.610299 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.611218 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.611409 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.611475 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.612828 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:22.612830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:22.612830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:22.612839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:22.614322 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.614738 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.614739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.614739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.614790 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:22.615031 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:22.615032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:22.615032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:22.615225 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:22.615337 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.615474 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.615550 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:40:22.617663 1390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:40:22.619925 1390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:40:22.627558 2138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.623108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.10.207.1:39873)
0807 08:40:22.623295 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.623300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.623311 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:40:22.623357 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.623359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.623376 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.623690 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:22.623761 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.624859 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.626428 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.627083 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.627181 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.627242 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.627423 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.627470 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
I20260807 08:40:22.627606 2135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.622898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39873 (local address 127.0.0.1:37022)
0807 08:40:22.623022 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.623062 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.623135 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.623507 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.623510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.623514 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:22.623604 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.623606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.624976 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.624977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.626328 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.626331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.627186 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.627187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.627288 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.627301 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.627404 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.627470 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260807 08:40:22.628309 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.622958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34759 (local address 127.0.0.1:57206)
0807 08:40:22.623785 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.623797 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.623843 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.624296 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.624297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.624299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:22.624346 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.624348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.625550 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.625552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.626684 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.626686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.628019 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.628020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.628106 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.628111 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.628176 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.628235 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":129,"threads_started":1}
I20260807 08:40:22.628312 2140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.623359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.10.207.2:33335)
0807 08:40:22.623606 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.623610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.623901 (+ 291us) server_negotiation.cc:407] Connection header received
0807 08:40:22.623966 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.623968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.623984 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.624070 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:22.624229 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.625053 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.626325 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.627132 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.627209 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.627274 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.627802 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.627853 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":121,"threads_started":1}
I20260807 08:40:22.628386 2139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.623263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.10.207.3:34759)
0807 08:40:22.623467 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.623492 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.624147 (+ 655us) server_negotiation.cc:407] Connection header received
0807 08:40:22.624197 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.624200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.624216 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.624296 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:22.624411 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.625427 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.626789 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.627577 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.627905 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.627974 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.628220 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.628295 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260807 08:40:22.628326 1583 tablet_service.cc:1467] Processing CreateTablet for tablet 78328bae2a9b430ea0b6814381b8f669 (DEFAULT_TABLE table=test-table [id=e0b9471716974829a49d79ebd24c1ab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:22.628572 2136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.622958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:33335 (local address 127.0.0.1:54788)
0807 08:40:22.623632 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.623645 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.623700 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.624084 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.624085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.624089 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:22.624144 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.624146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.625213 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.625214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.626231 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.626234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.627313 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.627314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.627382 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.627387 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.627443 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.627498 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":122,"threads_started":1}
I20260807 08:40:22.628711 1583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78328bae2a9b430ea0b6814381b8f669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:22.629292 2013 tablet_service.cc:1467] Processing CreateTablet for tablet 78328bae2a9b430ea0b6814381b8f669 (DEFAULT_TABLE table=test-table [id=e0b9471716974829a49d79ebd24c1ab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:22.629696 2013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78328bae2a9b430ea0b6814381b8f669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:22.631791 2141 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap starting.
I20260807 08:40:22.632300 1796 tablet_service.cc:1467] Processing CreateTablet for tablet 78328bae2a9b430ea0b6814381b8f669 (DEFAULT_TABLE table=test-table [id=e0b9471716974829a49d79ebd24c1ab1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:22.632522 2141 tablet_bootstrap.cc:654] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:22.632660 1796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78328bae2a9b430ea0b6814381b8f669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:22.633113 2141 log.cc:824] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:22.633167 2143 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Bootstrap starting.
I20260807 08:40:22.633985 2143 tablet_bootstrap.cc:654] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:22.634673 2143 log.cc:824] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:22.635402 2141 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: No bootstrap required, opened a new log
I20260807 08:40:22.635505 2141 ts_tablet_manager.cc:1397] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:40:22.635593 2143 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: No bootstrap required, opened a new log
I20260807 08:40:22.635696 2143 ts_tablet_manager.cc:1397] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:40:22.635934 2148 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Bootstrap starting.
I20260807 08:40:22.636689 2148 tablet_bootstrap.cc:654] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:22.637269 2148 log.cc:824] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:22.637526 2141 raft_consensus.cc:348] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.637765 2141 raft_consensus.cc:374] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:22.637815 2141 raft_consensus.cc:729] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9371bedbaa044a99fbd92052dd6630b, State: Initialized, Role: FOLLOWER
I20260807 08:40:22.637951 2141 consensus_queue.cc:260] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.638091 2143 raft_consensus.cc:348] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.638288 2143 raft_consensus.cc:374] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:22.638336 2148 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: No bootstrap required, opened a new log
I20260807 08:40:22.638329 2141 ts_tablet_manager.cc:1428] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:40:22.638339 2143 raft_consensus.cc:729] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e507b3fc1d7146e49f03c45f7c6976dc, State: Initialized, Role: FOLLOWER
I20260807 08:40:22.638410 2148 ts_tablet_manager.cc:1397] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:40:22.638417 1690 heartbeater.cc:503] Master 127.10.207.62:45045 was elected leader, sending a full tablet report...
I20260807 08:40:22.638501 2143 consensus_queue.cc:260] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.638952 2143 ts_tablet_manager.cc:1428] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 08:40:22.639087 2119 heartbeater.cc:503] Master 127.10.207.62:45045 was elected leader, sending a full tablet report...
I20260807 08:40:22.640506 2148 raft_consensus.cc:348] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.640637 2148 raft_consensus.cc:374] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:22.640661 2148 raft_consensus.cc:729] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7731efc76a9b4af6940a23e598c7a2af, State: Initialized, Role: FOLLOWER
I20260807 08:40:22.640793 2148 consensus_queue.cc:260] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.641083 2148 ts_tablet_manager.cc:1428] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:40:22.641161 1909 heartbeater.cc:503] Master 127.10.207.62:45045 was elected leader, sending a full tablet report...
W20260807 08:40:22.657241 1910 tablet.cc:2369] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:40:22.693215 2151 raft_consensus.cc:482] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:22.693393 2151 raft_consensus.cc:504] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.693913 2151 leader_election.cc:288] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9371bedbaa044a99fbd92052dd6630b (127.10.207.1:39873), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
I20260807 08:40:22.697882 2138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.694288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:38517 (local address 127.10.207.1:39873)
0807 08:40:22.694368 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.694371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.694382 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:40:22.694434 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.694435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.694451 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.694518 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:22.694896 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.695485 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.696795 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.697447 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.697534 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.697555 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.697737 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.697798 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:40:22.697875 2121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.694193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39873 (local address 127.10.207.3:38517)
0807 08:40:22.694287 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.694302 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.694353 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.694635 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.694636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.694639 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:22.694743 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.694746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.695606 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.695607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.696634 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.696636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.697556 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.697557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.697637 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.697642 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.697704 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.697755 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 08:40:22.698220 2140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.694318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:32973 (local address 127.10.207.2:33335)
0807 08:40:22.694396 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.694399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.694575 (+ 176us) server_negotiation.cc:407] Connection header received
0807 08:40:22.694628 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.694629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.694644 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.694723 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:22.695125 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.695715 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.697118 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.697718 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.697799 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.697807 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.698040 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.698136 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:40:22.698413 1615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78328bae2a9b430ea0b6814381b8f669" candidate_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9371bedbaa044a99fbd92052dd6630b" is_pre_election: true
I20260807 08:40:22.698519 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.694221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:33335 (local address 127.10.207.3:32973)
0807 08:40:22.694474 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.694483 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:22.694563 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:22.694872 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:22.694875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:22.694881 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:22.694991 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.694993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.695863 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.695864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.696989 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:22.696992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.697830 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.697831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:22.697903 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.697907 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:40:22.698366 (+ 459us) client_negotiation.cc:240] Negotiation successful
0807 08:40:22.698425 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":148,"threads_started":1}
I20260807 08:40:22.698652 1615 raft_consensus.cc:2454] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e507b3fc1d7146e49f03c45f7c6976dc in term 0.
I20260807 08:40:22.698911 1943 leader_election.cc:302] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e507b3fc1d7146e49f03c45f7c6976dc, e9371bedbaa044a99fbd92052dd6630b; no voters:
I20260807 08:40:22.699044 2151 raft_consensus.cc:2781] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:40:22.699106 2151 raft_consensus.cc:482] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:40:22.699136 2151 raft_consensus.cc:3037] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:22.699278 1831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78328bae2a9b430ea0b6814381b8f669" candidate_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7731efc76a9b4af6940a23e598c7a2af" is_pre_election: true
I20260807 08:40:22.699477 1831 raft_consensus.cc:2454] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e507b3fc1d7146e49f03c45f7c6976dc in term 0.
I20260807 08:40:22.700258 2151 raft_consensus.cc:504] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.700440 2151 leader_election.cc:288] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [CANDIDATE]: Term 1 election: Requested vote from peers e9371bedbaa044a99fbd92052dd6630b (127.10.207.1:39873), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
I20260807 08:40:22.700675 1615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78328bae2a9b430ea0b6814381b8f669" candidate_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9371bedbaa044a99fbd92052dd6630b"
I20260807 08:40:22.700692 1831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78328bae2a9b430ea0b6814381b8f669" candidate_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7731efc76a9b4af6940a23e598c7a2af"
I20260807 08:40:22.700778 1615 raft_consensus.cc:3037] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:22.700779 1831 raft_consensus.cc:3037] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:22.701589 1615 raft_consensus.cc:2454] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e507b3fc1d7146e49f03c45f7c6976dc in term 1.
I20260807 08:40:22.701593 1831 raft_consensus.cc:2454] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e507b3fc1d7146e49f03c45f7c6976dc in term 1.
I20260807 08:40:22.701839 1943 leader_election.cc:302] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e507b3fc1d7146e49f03c45f7c6976dc, e9371bedbaa044a99fbd92052dd6630b; no voters:
I20260807 08:40:22.701946 2151 raft_consensus.cc:2781] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:40:22.702167 2151 raft_consensus.cc:686] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 LEADER]: Becoming Leader. State: Replica: e507b3fc1d7146e49f03c45f7c6976dc, State: Running, Role: LEADER
I20260807 08:40:22.702291 2151 consensus_queue.cc:237] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:22.703356 1388 catalog_manager.cc:5495] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc reported cstate change: term changed from 0 to 1, leader changed from <none> to e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3). New cstate: current_term: 1 leader_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } health_report { overall_health: UNKNOWN } } }
I20260807 08:40:22.720021 2139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.717492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.10.207.3:34759)
0807 08:40:22.717540 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:22.717542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:22.717550 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:40:22.717587 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:22.717587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:22.717605 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:22.717686 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:40:22.717777 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.718286 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.719172 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:22.719386 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:22.719483 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:22.719645 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:40:22.719756 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:40:22.719878 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:22.719922 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 08:40:22.719946 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:40:22.722117 1831 raft_consensus.cc:1261] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Refusing update from remote peer e507b3fc1d7146e49f03c45f7c6976dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:22.722183 1615 raft_consensus.cc:1261] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Refusing update from remote peer e507b3fc1d7146e49f03c45f7c6976dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:22.722580 2155 consensus_queue.cc:1035] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:22.722749 2151 consensus_queue.cc:1035] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:22.725471 2165 mvcc.cc:205] Tried to move back new op lower bound from 7315832925066797056 to 7315832924989591552. Current Snapshot: MvccSnapshot[applied={T|T < 7315832925066797056}]
I20260807 08:40:22.725613 2164 mvcc.cc:205] Tried to move back new op lower bound from 7315832925066797056 to 7315832924989591552. Current Snapshot: MvccSnapshot[applied={T|T < 7315832925066797056}]
I20260807 08:40:22.728123 1390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:22.728221 1390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:22.729092 1390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Sending DeleteTablet for 3 replicas of tablet 78328bae2a9b430ea0b6814381b8f669
I20260807 08:40:22.729444 1583 tablet_service.cc:1514] Processing DeleteTablet for tablet 78328bae2a9b430ea0b6814381b8f669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37022
I20260807 08:40:22.729492 1796 tablet_service.cc:1514] Processing DeleteTablet for tablet 78328bae2a9b430ea0b6814381b8f669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:54788
I20260807 08:40:22.729707 2013 tablet_service.cc:1514] Processing DeleteTablet for tablet 78328bae2a9b430ea0b6814381b8f669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:57206
I20260807 08:40:22.729740 2174 tablet_replica.cc:330] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: stopping tablet replica
I20260807 08:40:22.729797 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 1490
I20260807 08:40:22.729936 2174 raft_consensus.cc:2229] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:22.730142 2174 raft_consensus.cc:2258] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:22.739802 2177 tablet_replica.cc:330] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: stopping tablet replica
I20260807 08:40:22.740024 2177 raft_consensus.cc:2229] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:22.740192 2177 raft_consensus.cc:2258] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:22.740419 2178 tablet_replica.cc:330] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: stopping tablet replica
I20260807 08:40:22.740535 2178 raft_consensus.cc:2229] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:40:22.740657 2178 raft_consensus.cc:2258] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:22.740964 2177 ts_tablet_manager.cc:1905] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:22.741201 2178 ts_tablet_manager.cc:1905] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:22.743188 2178 ts_tablet_manager.cc:1918] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:22.743287 2178 log.cc:1196] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/78328bae2a9b430ea0b6814381b8f669
I20260807 08:40:22.743464 2177 ts_tablet_manager.cc:1918] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:22.743604 2177 log.cc:1196] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/78328bae2a9b430ea0b6814381b8f669
I20260807 08:40:22.743606 2178 ts_tablet_manager.cc:1939] T 78328bae2a9b430ea0b6814381b8f669 P e507b3fc1d7146e49f03c45f7c6976dc: Deleting consensus metadata
I20260807 08:40:22.743958 2177 ts_tablet_manager.cc:1939] T 78328bae2a9b430ea0b6814381b8f669 P 7731efc76a9b4af6940a23e598c7a2af: Deleting consensus metadata
I20260807 08:40:22.744127 1366 catalog_manager.cc:4841] TS e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): tablet 78328bae2a9b430ea0b6814381b8f669 (table test-table [id=e0b9471716974829a49d79ebd24c1ab1]) successfully deleted
I20260807 08:40:22.744588 1365 catalog_manager.cc:4841] TS 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): tablet 78328bae2a9b430ea0b6814381b8f669 (table test-table [id=e0b9471716974829a49d79ebd24c1ab1]) successfully deleted
W20260807 08:40:22.745903 1362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:39873 (error 108)
I20260807 08:40:22.746191 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 1695
I20260807 08:40:22.750023 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:22.746339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39873 (local address 127.0.0.1:37028)
0807 08:40:22.749909 (+ 3570us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:22.749967 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:39873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3542}
W20260807 08:40:22.754398 1362 catalog_manager.cc:4584] TS e9371bedbaa044a99fbd92052dd6630b (127.10.207.1:39873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 78328bae2a9b430ea0b6814381b8f669: Network error: Client connection negotiation failed: client connection to 127.10.207.1:39873: connect: Connection refused (error 111)
I20260807 08:40:22.758826 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 1918
I20260807 08:40:22.770520 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 1338
I20260807 08:40:42.785236 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45045
--webserver_interface=127.10.207.62
--webserver_port=43593
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:45045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:40:42.878942 2181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:42.879222 2181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:42.879259 2181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:42.880605 2181 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:40:42.880668 2181 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:40:42.880693 2181 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:40:42.880723 2181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:40:42.880752 2181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:42.880781 2181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:42.880805 2181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:40:42.880836 2181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:40:42.882401 2181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:45045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=43593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:42.882723 2181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:42.882973 2181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:42.893899 2193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:42.893863 2191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:42.893863 2190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:42.894239 2181 server_base.cc:1034] running on GCE node
I20260807 08:40:42.894737 2181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:42.895140 2181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:42.896317 2181 hybrid_clock.cc:648] HybridClock initialized: now 1786092042896285 us; error 55 us; skew 500 ppm
Aug 07 08:40:42 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092042, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:42.899374 2181 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:40:42.899870 2181 webserver.cc:460] Webserver started at http://127.10.207.62:43593/ using document root <none> and password file <none>
I20260807 08:40:42.900153 2181 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:42.900221 2181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:42.901837 2181 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:42.902712 2203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:42.902984 2181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:42.903111 2181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "1434d9844f1f4b1f893a0fffb289ae34"
format_stamp: "Formatted at 2026-08-07 08:40:21 on dist-test-slave-h6n0"
server_key: "9a683d3bc4ff79fe2fd1e55e30a84c57"
server_key_iv: "214bb7414af029ca8dda88927902f9c2"
server_key_version: "encryptionkey@0"
I20260807 08:40:42.903559 2181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:42.944005 2181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:42.946477 2181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:42.946676 2181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:42.951790 2181 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:45045
I20260807 08:40:42.951861 2289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:45045 every 8 connection(s)
I20260807 08:40:42.952322 2181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:40:42.952842 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 2181
I20260807 08:40:42.953236 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:39873
--local_ip_for_outbound_sockets=127.10.207.1
--tserver_master_addrs=127.10.207.62:45045
--webserver_port=45111
--webserver_interface=127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:42.954644 2290 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:40:42.955605 2290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Bootstrap starting.
I20260807 08:40:42.967878 2290 log.cc:824] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:42.970391 2290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:40:42.970849 2290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Bootstrap complete.
I20260807 08:40:42.973582 2290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:42.973857 2290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1434d9844f1f4b1f893a0fffb289ae34, State: Initialized, Role: FOLLOWER
I20260807 08:40:42.974061 2290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:42.974139 2290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:40:42.974202 2290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:40:42.974259 2290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:40:42.975544 2290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:42.975648 2290 leader_election.cc:302] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1434d9844f1f4b1f893a0fffb289ae34; no voters:
I20260807 08:40:42.975921 2290 leader_election.cc:288] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:40:42.975966 2295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:40:42.976128 2295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 LEADER]: Becoming Leader. State: Replica: 1434d9844f1f4b1f893a0fffb289ae34, State: Running, Role: LEADER
I20260807 08:40:42.976289 2290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:40:42.976315 2295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:40:42.976754 2297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1434d9844f1f4b1f893a0fffb289ae34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } } }
I20260807 08:40:42.976841 2297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:42.977240 2298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1434d9844f1f4b1f893a0fffb289ae34. Latest consensus state: current_term: 2 leader_uuid: "1434d9844f1f4b1f893a0fffb289ae34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } } }
I20260807 08:40:42.977388 2298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: This master's current role is: LEADER
I20260807 08:40:42.977650 2306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:40:42.978608 2306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:40:42.978873 2306 catalog_manager.cc:1259] Loaded cluster ID: 5ddf985dde9d49d0bb1401d9c073355c
I20260807 08:40:42.979110 2306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:40:42.980397 2306 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:40:42.980620 2306 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Loaded TSK: 0
I20260807 08:40:42.995050 2306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Generated new TSK 1
I20260807 08:40:42.995195 2306 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:40:43.045068 2292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:43.045289 2292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:43.045331 2292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:43.046419 2292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:43.046486 2292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:43.046558 2292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:40:43.047986 2292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:39873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=45111
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:43.048240 2292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:43.048465 2292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:43.055095 2340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:43.055189 2334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:43.055699 2335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:43.055786 2292 server_base.cc:1034] running on GCE node
I20260807 08:40:43.056080 2292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:43.056401 2292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:43.057595 2292 hybrid_clock.cc:648] HybridClock initialized: now 1786092043057553 us; error 60 us; skew 500 ppm
Aug 07 08:40:43 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:43.061144 2292 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:40:43.061671 2292 webserver.cc:460] Webserver started at http://127.10.207.1:45111/ using document root <none> and password file <none>
I20260807 08:40:43.061911 2292 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:43.061995 2292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:43.063578 2292 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:43.064433 2353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:43.064635 2292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:43.064728 2292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "e9371bedbaa044a99fbd92052dd6630b"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d47ededd34ecb05435b18a72afba18b6"
server_key_iv: "e0c84ed29890f66186480a16c73097d9"
server_key_version: "encryptionkey@0"
I20260807 08:40:43.065080 2292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:43.083570 2292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:43.085692 2292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:43.085839 2292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:43.086114 2292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:43.086613 2367 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:40:43.088080 2292 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:40:43.088140 2292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:40:43.088163 2292 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:40:43.088815 2292 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:40:43.088864 2292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:43.088901 2367 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap starting.
I20260807 08:40:43.097771 2292 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:39873
I20260807 08:40:43.098258 2292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:40:43.099001 2548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:39873 every 8 connection(s)
I20260807 08:40:43.100764 2367 log.cc:824] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:43.103039 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 2292
I20260807 08:40:43.103192 2367 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:40:43.103427 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:33335
--local_ip_for_outbound_sockets=127.10.207.2
--tserver_master_addrs=127.10.207.62:45045
--webserver_port=44075
--webserver_interface=127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:43.103519 2367 tablet_bootstrap.cc:492] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap complete.
I20260807 08:40:43.104809 2367 ts_tablet_manager.cc:1397] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.012s
I20260807 08:40:43.106163 2367 raft_consensus.cc:348] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:43.106480 2367 raft_consensus.cc:729] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9371bedbaa044a99fbd92052dd6630b, State: Initialized, Role: FOLLOWER
I20260807 08:40:43.107008 2367 consensus_queue.cc:260] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } } peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } }
I20260807 08:40:43.107388 2367 ts_tablet_manager.cc:1428] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
Aug 07 08:40:43 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:43.113951 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.099618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:51267 (local address 127.10.207.62:45045)
0807 08:40:43.100873 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.100882 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.100915 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:40:43.101070 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.101077 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.101135 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.101270 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:43.102334 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.103860 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.105481 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.106112 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.106221 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.109389 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:43.109395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:43.109396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:43.109412 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:43.111741 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.112321 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.112324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.112325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.112442 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.112771 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.112772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.112773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.113080 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:43.113169 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.113562 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.113719 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":97,"threads_started":1}
I20260807 08:40:43.114069 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.099144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:51267)
0807 08:40:43.099845 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.099896 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.100662 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.101421 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.101426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.101842 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:43.102133 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.102140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.104059 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.104061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.105372 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.105376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.106313 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.106315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.106516 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.107179 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:43.107188 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:43.109251 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:43.111853 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.111862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.111864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.112216 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.112532 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.112533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.112534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.112685 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.113173 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:43.113182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:43.113356 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.113490 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.113680 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":132,"threads_started":1}
I20260807 08:40:43.115526 2549 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45045
I20260807 08:40:43.115664 2549 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:43.115878 2549 heartbeater.cc:511] Master 127.10.207.62:45045 requested a full tablet report, sending...
I20260807 08:40:43.116909 2230 ts_manager.cc:194] Registered new tserver with Master: e9371bedbaa044a99fbd92052dd6630b (127.10.207.1:39873)
I20260807 08:40:43.118203 2230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:51267
I20260807 08:40:43.119360 2549 heartbeater.cc:503] Master 127.10.207.62:45045 was elected leader, sending a full tablet report...
Aug 07 08:40:43 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092042, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.1@KRBTEST.COM
I20260807 08:40:43.128883 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.117665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39873 (local address 127.0.0.1:37862)
0807 08:40:43.118591 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.118629 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.118707 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.119027 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.119032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.119460 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:43.119566 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.119569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.121060 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.121061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.122163 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.122166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.122958 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.122959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.123053 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.123626 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:43.123632 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:43.125310 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:43.127537 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.127544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.127546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.127837 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.128088 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.128089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.128089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.128162 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.128484 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:43.128491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:43.128545 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.128639 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.128773 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":127,"threads_started":1}
I20260807 08:40:43.128919 2561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.117977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37862 (local address 127.10.207.1:39873)
0807 08:40:43.118852 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.118857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.118867 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:40:43.118906 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.118908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.118925 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.118989 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:43.119893 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.120962 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.122268 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.122864 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.122946 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.125415 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:43.125423 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:43.125423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:43.125431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:43.127435 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.127936 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.127938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.127939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.128003 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.128237 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.128237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.128238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.128398 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:43.128494 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.128714 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.128808 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":95,"threads_started":1}
I20260807 08:40:43.129511 2411 tablet_service.cc:1514] Processing DeleteTablet for tablet 78328bae2a9b430ea0b6814381b8f669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:22 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37862
I20260807 08:40:43.130184 2562 tablet_replica.cc:330] stopping tablet replica
I20260807 08:40:43.130311 2562 raft_consensus.cc:2229] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:43.130393 2562 raft_consensus.cc:2258] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:43.130841 2562 ts_tablet_manager.cc:1905] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:43.133486 2562 ts_tablet_manager.cc:1918] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:43.133581 2562 log.cc:1196] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal/wals/78328bae2a9b430ea0b6814381b8f669
I20260807 08:40:43.133846 2562 ts_tablet_manager.cc:1939] T 78328bae2a9b430ea0b6814381b8f669 P e9371bedbaa044a99fbd92052dd6630b: Deleting consensus metadata
I20260807 08:40:43.134506 2205 catalog_manager.cc:4841] TS e9371bedbaa044a99fbd92052dd6630b (127.10.207.1:39873): tablet 78328bae2a9b430ea0b6814381b8f669 (table test-table [id=e0b9471716974829a49d79ebd24c1ab1]) successfully deleted
W20260807 08:40:43.197829 2557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:43.198057 2557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:43.198096 2557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:43.199222 2557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:43.199290 2557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:43.199337 2557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:40:43.200786 2557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:33335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=44075
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:43.201045 2557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:43.201293 2557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:40:43.206656 2557 server_base.cc:1034] running on GCE node
W20260807 08:40:43.206730 2572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:43.206568 2569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:43.206588 2568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:43.207127 2557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:43.207427 2557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:43.208587 2557 hybrid_clock.cc:648] HybridClock initialized: now 1786092043208586 us; error 41 us; skew 500 ppm
Aug 07 08:40:43 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:43.211176 2557 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:40:43.211640 2557 webserver.cc:460] Webserver started at http://127.10.207.2:44075/ using document root <none> and password file <none>
I20260807 08:40:43.211869 2557 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:43.211930 2557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:43.213339 2557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:43.214017 2583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:43.214236 2557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:43.214344 2557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "7731efc76a9b4af6940a23e598c7a2af"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "34e1773f8ebf322e327b30edeb5427a1"
server_key_iv: "80b56acf964ae7a2bebb21e4a760b391"
server_key_version: "encryptionkey@0"
I20260807 08:40:43.214840 2557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:43.242254 2557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:43.244699 2557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:43.244837 2557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:43.245085 2557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:43.245471 2557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:43.245504 2557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:43.245529 2557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:43.245582 2557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:43.253196 2557 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:33335
I20260807 08:40:43.253328 2757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:33335 every 8 connection(s)
I20260807 08:40:43.253686 2557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:40:43 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:43.262410 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 2557
I20260807 08:40:43.262843 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:34759
--local_ip_for_outbound_sockets=127.10.207.3
--tserver_master_addrs=127.10.207.62:45045
--webserver_port=35157
--webserver_interface=127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:40:43.264925 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.254639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:48725 (local address 127.10.207.62:45045)
0807 08:40:43.254721 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.254723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.255021 (+ 298us) server_negotiation.cc:407] Connection header received
0807 08:40:43.255595 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.255596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.255613 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.255662 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:43.256690 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.257212 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.258221 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.258430 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.258519 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.261144 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:43.261146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:43.261147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:43.261156 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:43.263006 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.263697 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.263698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.263698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.263778 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.264139 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.264140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.264140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.264261 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:43.264331 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.264754 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.264849 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:40:43.265292 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.254609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:48725)
0807 08:40:43.254847 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.254902 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.255483 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.255752 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.255761 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.256380 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:43.256587 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.256592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.257317 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.257318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.258129 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.258132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.258521 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.258522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.258673 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.259237 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:43.259245 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:43.260943 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:43.263137 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.263150 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.263153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.263581 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.263875 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.263876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.263877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.264052 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.264343 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:43.264352 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:43.264561 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.264700 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.264937 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
I20260807 08:40:43.266008 2760 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45045
I20260807 08:40:43.266136 2760 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:43.266347 2760 heartbeater.cc:511] Master 127.10.207.62:45045 requested a full tablet report, sending...
I20260807 08:40:43.266921 2230 ts_manager.cc:194] Registered new tserver with Master: 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
I20260807 08:40:43.267357 2230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:48725
W20260807 08:40:43.367893 2763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:40:43.368269 2763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:40:43.368315 2763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:40:43.370517 2763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:40:43.370602 2763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:40:43.370646 2763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:40:43.373210 2763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:34759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=35157
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:40:43.373477 2763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:40:43.373749 2763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:40:43.379793 2776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:43.379796 2770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:40:43.379799 2769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:40:43.380653 2763 server_base.cc:1034] running on GCE node
I20260807 08:40:43.380892 2763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:40:43.381229 2763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:40:43.382404 2763 hybrid_clock.cc:648] HybridClock initialized: now 1786092043382393 us; error 29 us; skew 500 ppm
Aug 07 08:40:43 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:40:43.385401 2763 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:40:43.385952 2763 webserver.cc:460] Webserver started at http://127.10.207.3:35157/ using document root <none> and password file <none>
I20260807 08:40:43.386216 2763 fs_manager.cc:359] Metadata directory not provided
I20260807 08:40:43.386315 2763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:40:43.387974 2763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:43.388782 2788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:43.389062 2763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:43.389222 2763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "e507b3fc1d7146e49f03c45f7c6976dc"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d6ca6beeabb254321550eeb63a9f126b"
server_key_iv: "1b76a43e440e947e315591c72c303799"
server_key_version: "encryptionkey@0"
I20260807 08:40:43.389875 2763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:40:43.424007 2763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:40:43.426879 2763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:40:43.427064 2763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:40:43.427376 2763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:40:43.427781 2763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:40:43.427837 2763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:43.427873 2763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:40:43.427909 2763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:40:43.437273 2763 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34759
I20260807 08:40:43.437382 2960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34759 every 8 connection(s)
I20260807 08:40:43.437805 2763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:40:43.442251 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 2763
Aug 07 08:40:43 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:40:43.450644 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.438856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:43575 (local address 127.10.207.62:45045)
0807 08:40:43.438948 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.438951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.439222 (+ 271us) server_negotiation.cc:407] Connection header received
0807 08:40:43.439816 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.439818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.439832 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.439952 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:43.441056 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.441936 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.443659 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.443907 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.443969 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.447131 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:43.447132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:43.447132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:43.447141 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:43.449056 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.449492 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.449494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.449494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.449560 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.449842 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.449843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.449843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.449968 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:43.450049 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.450433 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.450518 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:40:43.450891 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.438816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:43575)
0807 08:40:43.439051 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.439100 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.439699 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.440016 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.440020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.440694 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:40:43.440935 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.440940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.442087 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.442089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.443561 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.443564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.444007 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.444008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.444180 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.444841 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:40:43.444852 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:40:43.447023 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:40:43.449150 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.449158 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.449161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.449407 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.449641 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:40:43.449642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:40:43.449643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:40:43.449763 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:40:43.450041 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:40:43.450048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:40:43.450268 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.450409 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.450631 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":130,"threads_started":1}
I20260807 08:40:43.451399 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.443098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44168 (local address 127.10.207.62:45045)
0807 08:40:43.443283 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.443286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.443298 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:40:43.443323 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.443324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.443337 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.443439 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:43.444283 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.444912 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.446009 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.446353 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.446460 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.448038 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:43.448145 (+ 107us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:43.448147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:43.448161 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:43.449828 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.450255 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.450257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.450257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.450328 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.450588 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.450589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.450590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.450746 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:43.450957 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.451171 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.451277 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260807 08:40:43.451526 2961 heartbeater.cc:347] Connected to a master server at 127.10.207.62:45045
I20260807 08:40:43.451680 2961 heartbeater.cc:464] Registering TS with master...
I20260807 08:40:43.451928 2961 heartbeater.cc:511] Master 127.10.207.62:45045 requested a full tablet report, sending...
I20260807 08:40:43.452342 2230 ts_manager.cc:194] Registered new tserver with Master: e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759)
I20260807 08:40:43.452682 2230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:43575
I20260807 08:40:43.463536 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:40:43.466833 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.464042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.10.207.62:45045)
0807 08:40:43.464121 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.464124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.464277 (+ 153us) server_negotiation.cc:407] Connection header received
0807 08:40:43.464388 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.464389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.464433 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.464521 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:40:43.464684 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.465165 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.465987 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.466190 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.466271 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.466481 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:40:43.466574 (+ 93us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 08:40:43.466761 (+ 187us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44174: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 08:40:43.466924 2966 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44174: token expired
W20260807 08:40:43.467692 2122 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.207.62:45045)
I20260807 08:40:43.474314 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.468170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.10.207.62:45045)
0807 08:40:43.468241 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.468243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.468251 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:40:43.468361 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.468362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.468374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.468506 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:40:43.469106 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.469608 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.470385 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.470675 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.470746 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.472094 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:40:43.472095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:40:43.472095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:40:43.472102 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:40:43.473348 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.473688 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.473689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.473689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.473739 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:40:43.473896 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:40:43.473897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:40:43.473897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:40:43.473990 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:40:43.474070 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.474202 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.474260 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:40:43.475813 2230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:40:43.477767 2230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:40:43.480674 2411 tablet_service.cc:1467] Processing CreateTablet for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 (DEFAULT_TABLE table=test-table [id=c19f92a7afd44720bbbedc1fa159d393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:43.480924 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:43.482678 2367 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap starting.
I20260807 08:40:43.483058 2367 tablet_bootstrap.cc:654] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:43.483593 2367 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: No bootstrap required, opened a new log
I20260807 08:40:43.483659 2367 ts_tablet_manager.cc:1397] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:40:43.483978 2367 raft_consensus.cc:348] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.484064 2367 raft_consensus.cc:374] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:43.484120 2367 raft_consensus.cc:729] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9371bedbaa044a99fbd92052dd6630b, State: Initialized, Role: FOLLOWER
I20260807 08:40:43.484182 2367 consensus_queue.cc:260] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.484293 2367 ts_tablet_manager.cc:1428] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:40:43.484791 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.480572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.10.207.3:34759)
0807 08:40:43.480813 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.480827 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.480844 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:40:43.480899 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.480901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.480919 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.481116 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:43.481301 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.482182 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.483528 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.484211 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.484292 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.484366 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.484635 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.484712 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
I20260807 08:40:43.484773 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.480357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34759 (local address 127.0.0.1:38732)
0807 08:40:43.480412 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.480426 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.480464 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.481124 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.481125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.481127 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:43.481197 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.481199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.482284 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.482285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.483420 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.483423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.484425 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.484427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.484510 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.484533 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.484600 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.484666 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:40:43.486048 2869 tablet_service.cc:1467] Processing CreateTablet for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 (DEFAULT_TABLE table=test-table [id=c19f92a7afd44720bbbedc1fa159d393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:43.486474 2869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:43.487027 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.480413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:33335 (local address 127.0.0.1:47560)
0807 08:40:43.480530 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.480544 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.480608 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.481544 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.481546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.481549 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:43.481596 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.481597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.484594 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.484595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.485727 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.485730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.486732 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.486733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.486813 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.486818 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.486894 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.486952 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260807 08:40:43.487062 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.480712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.10.207.2:33335)
0807 08:40:43.481353 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.481359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.481372 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:40:43.481414 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.481416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.481432 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.481525 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:43.481666 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.484492 (+ 2826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.485857 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.486535 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.486732 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.486830 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.486907 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.486973 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":110,"threads_started":1}
I20260807 08:40:43.490088 2975 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Bootstrap starting.
I20260807 08:40:43.490895 2975 tablet_bootstrap.cc:654] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:43.490921 2662 tablet_service.cc:1467] Processing CreateTablet for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 (DEFAULT_TABLE table=test-table [id=c19f92a7afd44720bbbedc1fa159d393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:40:43.491326 2662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:40:43.491515 2975 log.cc:824] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:43.492414 2975 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: No bootstrap required, opened a new log
I20260807 08:40:43.492486 2975 ts_tablet_manager.cc:1397] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:40:43.494408 2975 raft_consensus.cc:348] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.494652 2975 raft_consensus.cc:374] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:43.494700 2975 raft_consensus.cc:729] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e507b3fc1d7146e49f03c45f7c6976dc, State: Initialized, Role: FOLLOWER
I20260807 08:40:43.494762 2977 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Bootstrap starting.
I20260807 08:40:43.494853 2975 consensus_queue.cc:260] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.495177 2975 ts_tablet_manager.cc:1428] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 08:40:43.495244 2961 heartbeater.cc:503] Master 127.10.207.62:45045 was elected leader, sending a full tablet report...
I20260807 08:40:43.495579 2977 tablet_bootstrap.cc:654] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Neither blocks nor log segments found. Creating new log.
I20260807 08:40:43.496358 2977 log.cc:824] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Log is configured to *not* fsync() on all Append() calls
I20260807 08:40:43.497398 2977 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: No bootstrap required, opened a new log
I20260807 08:40:43.497483 2977 ts_tablet_manager.cc:1397] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:40:43.499015 2977 raft_consensus.cc:348] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.499193 2977 raft_consensus.cc:374] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:40:43.499243 2977 raft_consensus.cc:729] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7731efc76a9b4af6940a23e598c7a2af, State: Initialized, Role: FOLLOWER
I20260807 08:40:43.499392 2977 consensus_queue.cc:260] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.499701 2977 ts_tablet_manager.cc:1428] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:40:43.499807 2760 heartbeater.cc:503] Master 127.10.207.62:45045 was elected leader, sending a full tablet report...
W20260807 08:40:43.504317 2761 tablet.cc:2369] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:40:43.547596 2558 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:40:43.547770 2558 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.548215 2558 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
I20260807 08:40:43.551542 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.548405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34759 (local address 127.10.207.1:59147)
0807 08:40:43.548488 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.548502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.548581 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.548855 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.548856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.548859 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:43.548935 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.548937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.549713 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.549714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.550645 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.550648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.551271 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.551272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.551353 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.551381 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.551439 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.551476 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 08:40:43.551662 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.548539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:59147 (local address 127.10.207.3:34759)
0807 08:40:43.548597 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.548599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.548613 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:40:43.548685 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.548686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.548698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.548783 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:43.549022 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.549627 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.550741 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.551180 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.551243 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.551253 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.551488 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.551547 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:40:43.551764 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.548566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:42609 (local address 127.10.207.2:33335)
0807 08:40:43.548620 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.548622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.548874 (+ 252us) server_negotiation.cc:407] Connection header received
0807 08:40:43.548913 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.548915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.548931 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.548997 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:40:43.549238 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.549752 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.550854 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.551264 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.551347 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.551356 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.551628 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.551674 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:40:43.552034 2897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e66ab9dcf0b4534b8d2ec2da31fcc39" candidate_uuid: "e9371bedbaa044a99fbd92052dd6630b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" is_pre_election: true
I20260807 08:40:43.552258 2897 raft_consensus.cc:2454] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9371bedbaa044a99fbd92052dd6630b in term 0.
I20260807 08:40:43.552398 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.548493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:33335 (local address 127.10.207.1:42609)
0807 08:40:43.548718 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.548731 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:40:43.548812 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:40:43.549024 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:40:43.549025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:40:43.549027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:40:43.549128 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.549130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.549845 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.549846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.550764 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:40:43.550767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.551391 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.551393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:40:43.551521 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.551527 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:40:43.551606 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 08:40:43.551764 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":117,"threads_started":1}
I20260807 08:40:43.552583 2358 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e507b3fc1d7146e49f03c45f7c6976dc, e9371bedbaa044a99fbd92052dd6630b; no voters:
I20260807 08:40:43.552786 2558 raft_consensus.cc:2781] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:40:43.552860 2558 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:40:43.552805 2692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e66ab9dcf0b4534b8d2ec2da31fcc39" candidate_uuid: "e9371bedbaa044a99fbd92052dd6630b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7731efc76a9b4af6940a23e598c7a2af" is_pre_election: true
I20260807 08:40:43.552886 2558 raft_consensus.cc:3037] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:43.552966 2692 raft_consensus.cc:2454] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9371bedbaa044a99fbd92052dd6630b in term 0.
I20260807 08:40:43.553680 2558 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.553853 2558 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 1 election: Requested vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
I20260807 08:40:43.554019 2692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e66ab9dcf0b4534b8d2ec2da31fcc39" candidate_uuid: "e9371bedbaa044a99fbd92052dd6630b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7731efc76a9b4af6940a23e598c7a2af"
I20260807 08:40:43.554054 2897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e66ab9dcf0b4534b8d2ec2da31fcc39" candidate_uuid: "e9371bedbaa044a99fbd92052dd6630b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e507b3fc1d7146e49f03c45f7c6976dc"
I20260807 08:40:43.554133 2692 raft_consensus.cc:3037] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:43.554172 2897 raft_consensus.cc:3037] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:40:43.555008 2897 raft_consensus.cc:2454] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9371bedbaa044a99fbd92052dd6630b in term 1.
I20260807 08:40:43.555212 2358 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e507b3fc1d7146e49f03c45f7c6976dc, e9371bedbaa044a99fbd92052dd6630b; no voters:
I20260807 08:40:43.555262 2692 raft_consensus.cc:2454] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9371bedbaa044a99fbd92052dd6630b in term 1.
I20260807 08:40:43.555321 2558 raft_consensus.cc:2781] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:40:43.555521 2558 raft_consensus.cc:686] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 LEADER]: Becoming Leader. State: Replica: e9371bedbaa044a99fbd92052dd6630b, State: Running, Role: LEADER
I20260807 08:40:43.555639 2558 consensus_queue.cc:237] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:40:43.556522 2230 catalog_manager.cc:5495] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b reported cstate change: term changed from 0 to 1, leader changed from <none> to e9371bedbaa044a99fbd92052dd6630b (127.10.207.1). New cstate: current_term: 1 leader_uuid: "e9371bedbaa044a99fbd92052dd6630b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } health_report { overall_health: HEALTHY } } }
I20260807 08:40:43.563056 2561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.560630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37868 (local address 127.10.207.1:39873)
0807 08:40:43.560712 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 08:40:43.560714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:40:43.560748 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:40:43.560819 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:40:43.560820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:40:43.560838 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:40:43.560906 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:40:43.561044 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.561542 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.562232 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:40:43.562433 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:40:43.562487 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:40:43.562693 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:40:43.562804 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:40:43.562869 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:40:43.562928 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 08:40:43.562952 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:40:43.564780 2692 raft_consensus.cc:1261] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Refusing update from remote peer e9371bedbaa044a99fbd92052dd6630b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:43.564829 2897 raft_consensus.cc:1261] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 FOLLOWER]: Refusing update from remote peer e9371bedbaa044a99fbd92052dd6630b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:40:43.565183 2984 consensus_queue.cc:1035] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:43.565327 2558 consensus_queue.cc:1035] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:40:43.567943 2992 mvcc.cc:205] Tried to move back new op lower bound from 7315833010438688768 to 7315833010404352000. Current Snapshot: MvccSnapshot[applied={T|T < 7315833010438688768}]
I20260807 08:40:43.569057 2996 mvcc.cc:205] Tried to move back new op lower bound from 7315833010438688768 to 7315833010404352000. Current Snapshot: MvccSnapshot[applied={T|T < 7315833010438688768 or (T in {7315833010438688768})}]
I20260807 08:40:43.569226 2994 mvcc.cc:205] Tried to move back new op lower bound from 7315833010438688768 to 7315833010404352000. Current Snapshot: MvccSnapshot[applied={T|T < 7315833010438688768 or (T in {7315833010438688768})}]
I20260807 08:40:43.571417 2230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:43.571527 2230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:40:43.572477 2230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Sending DeleteTablet for 3 replicas of tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:40:43.572875 2662 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:43 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:47560
I20260807 08:40:43.572930 2869 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:43 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:38732
I20260807 08:40:43.573123 2411 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:40:43 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37862
I20260807 08:40:43.573194 3011 tablet_replica.cc:330] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: stopping tablet replica
I20260807 08:40:43.573226 2562 tablet_replica.cc:330] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: stopping tablet replica
I20260807 08:40:43.573215 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 2292
I20260807 08:40:43.573279 2562 raft_consensus.cc:2229] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:40:43.573307 3011 raft_consensus.cc:2229] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:43.573385 2562 raft_consensus.cc:2258] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:43.573421 3011 raft_consensus.cc:2258] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:43.573589 3007 tablet_replica.cc:330] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: stopping tablet replica
I20260807 08:40:43.573683 3007 raft_consensus.cc:2229] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:40:43.573762 3007 raft_consensus.cc:2258] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:40:43.578752 3011 ts_tablet_manager.cc:1905] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:43.579487 3007 ts_tablet_manager.cc:1905] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:40:43.581228 3007 ts_tablet_manager.cc:1918] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:43.581326 3007 log.cc:1196] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:40:43.581640 3007 ts_tablet_manager.cc:1939] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P 7731efc76a9b4af6940a23e598c7a2af: Deleting consensus metadata
I20260807 08:40:43.581624 3011 ts_tablet_manager.cc:1918] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:40:43.581740 3011 log.cc:1196] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:40:43.582100 2211 catalog_manager.cc:4841] TS 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 (table test-table [id=c19f92a7afd44720bbbedc1fa159d393]) successfully deleted
I20260807 08:40:43.582150 3011 ts_tablet_manager.cc:1939] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e507b3fc1d7146e49f03c45f7c6976dc: Deleting consensus metadata
I20260807 08:40:43.582800 2212 catalog_manager.cc:4841] TS e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39 (table test-table [id=c19f92a7afd44720bbbedc1fa159d393]) successfully deleted
W20260807 08:40:43.592931 2205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:39873 (error 108)
I20260807 08:40:43.593362 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 2557
I20260807 08:40:43.593623 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:40:43.593408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39873 (local address 127.0.0.1:37880)
0807 08:40:43.593516 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:40:43.593576 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:39873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 08:40:43.599260 2205 catalog_manager.cc:4584] TS e9371bedbaa044a99fbd92052dd6630b (127.10.207.1:39873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e66ab9dcf0b4534b8d2ec2da31fcc39: Network error: Client connection negotiation failed: client connection to 127.10.207.1:39873: connect: Connection refused (error 111)
I20260807 08:40:43.604562 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 2763
I20260807 08:40:43.615375 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 2181
I20260807 08:41:03.628657 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45045
--webserver_interface=127.10.207.62
--webserver_port=43593
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:45045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:03.721623 3013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:03.721915 3013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:03.721952 3013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:03.723156 3013 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:41:03.723228 3013 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:41:03.723258 3013 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:41:03.723279 3013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:03.723311 3013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:03.723337 3013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:03.723366 3013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:03.723395 3013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:03.724982 3013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:45045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=43593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:03.725339 3013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:03.725601 3013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:03.731616 3024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:03.731734 3013 server_base.cc:1034] running on GCE node
W20260807 08:41:03.731573 3026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:03.731858 3022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:03.732221 3013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:03.732532 3013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:03.733690 3013 hybrid_clock.cc:648] HybridClock initialized: now 1786092063733669 us; error 40 us; skew 500 ppm
Aug 07 08:41:03 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092063, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:03.736678 3013 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:03.737192 3013 webserver.cc:460] Webserver started at http://127.10.207.62:43593/ using document root <none> and password file <none>
I20260807 08:41:03.737469 3013 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:03.737524 3013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:03.739097 3013 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:41:03.739950 3038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:03.740206 3013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:03.740334 3013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "1434d9844f1f4b1f893a0fffb289ae34"
format_stamp: "Formatted at 2026-08-07 08:40:21 on dist-test-slave-h6n0"
server_key: "9a683d3bc4ff79fe2fd1e55e30a84c57"
server_key_iv: "214bb7414af029ca8dda88927902f9c2"
server_key_version: "encryptionkey@0"
I20260807 08:41:03.740736 3013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:03.761965 3013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:03.763931 3013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:03.764101 3013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:03.768985 3013 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:45045
I20260807 08:41:03.769068 3122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:45045 every 8 connection(s)
I20260807 08:41:03.769454 3013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:03.771325 3125 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:41:03.772131 3125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Bootstrap starting.
I20260807 08:41:03.776247 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 3013
I20260807 08:41:03.776597 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:39873
--local_ip_for_outbound_sockets=127.10.207.1
--tserver_master_addrs=127.10.207.62:45045
--webserver_port=45111
--webserver_interface=127.10.207.1
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:41:03.784785 3125 log.cc:824] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:03.788659 3125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:41:03.789079 3125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Bootstrap complete.
I20260807 08:41:03.793067 3125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:41:03.793412 3125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1434d9844f1f4b1f893a0fffb289ae34, State: Initialized, Role: FOLLOWER
I20260807 08:41:03.793591 3125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:41:03.793663 3125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:03.793700 3125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:03.793751 3125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:41:03.795230 3125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:41:03.795353 3125 leader_election.cc:302] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1434d9844f1f4b1f893a0fffb289ae34; no voters:
I20260807 08:41:03.795560 3125 leader_election.cc:288] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:41:03.795670 3130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:41:03.795863 3130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [term 3 LEADER]: Becoming Leader. State: Replica: 1434d9844f1f4b1f893a0fffb289ae34, State: Running, Role: LEADER
I20260807 08:41:03.795874 3125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:03.796015 3130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } }
I20260807 08:41:03.796564 3133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1434d9844f1f4b1f893a0fffb289ae34. Latest consensus state: current_term: 3 leader_uuid: "1434d9844f1f4b1f893a0fffb289ae34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } } }
I20260807 08:41:03.796685 3133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:03.797079 3145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:03.797247 3132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1434d9844f1f4b1f893a0fffb289ae34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434d9844f1f4b1f893a0fffb289ae34" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45045 } } }
I20260807 08:41:03.797338 3132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:03.798027 3145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:03.798135 3145 catalog_manager.cc:1259] Loaded cluster ID: 5ddf985dde9d49d0bb1401d9c073355c
I20260807 08:41:03.798159 3145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:03.799511 3145 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:03.799723 3145 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Loaded TSK: 0
I20260807 08:41:03.799777 3145 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1434d9844f1f4b1f893a0fffb289ae34: Loaded TSK: 1
F20260807 08:41:03.800056 3145 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa89bdf99ad google::LogMessage::Fail() at ??:0
@ 0x7fa89bdfc88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa89bdf94de google::LogMessage::Flush() at ??:0
@ 0x7fa89bdfaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa89e566104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa89e563d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa8a3d531cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa8a3d5348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa8a3d53594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa8a3d19c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa8a3d23ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa89dcc70d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa89dcbc954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa89eb206db start_thread at ??:0
@ 0x7fa89a8f761f clone at ??:0
W20260807 08:41:03.902626 3127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:03.902877 3127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:03.902907 3127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:03.904245 3127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:03.904295 3127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:03.904326 3127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:41:03.905962 3127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:39873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=45111
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:03.906250 3127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:03.906569 3127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:03.912365 3193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:03.912372 3190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:03.912494 3191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:03.913200 3127 server_base.cc:1034] running on GCE node
I20260807 08:41:03.913410 3127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:03.913724 3127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:03.914881 3127 hybrid_clock.cc:648] HybridClock initialized: now 1786092063914871 us; error 30 us; skew 500 ppm
Aug 07 08:41:03 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092063, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:03.917919 3127 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:41:03.918406 3127 webserver.cc:460] Webserver started at http://127.10.207.1:45111/ using document root <none> and password file <none>
I20260807 08:41:03.918757 3127 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:03.918834 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:03.920437 3127 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:03.921363 3202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:03.921643 3127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:03.921765 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "e9371bedbaa044a99fbd92052dd6630b"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d47ededd34ecb05435b18a72afba18b6"
server_key_iv: "e0c84ed29890f66186480a16c73097d9"
server_key_version: "encryptionkey@0"
I20260807 08:41:03.922219 3127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:03.933589 3127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:03.936188 3127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:03.936352 3127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:03.936627 3127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:03.937170 3213 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:41:03.939337 3127 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:41:03.939419 3127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:41:03.939471 3127 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:41:03.940307 3127 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:41:03.940372 3127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:41:03.940482 3213 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap starting.
I20260807 08:41:03.949282 3127 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:39873
I20260807 08:41:03.949438 3369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:39873 every 8 connection(s)
I20260807 08:41:03.949783 3127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:41:03.953297 3213 log.cc:824] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:03.954851 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 3127
I20260807 08:41:03.955171 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:33335
--local_ip_for_outbound_sockets=127.10.207.2
--tserver_master_addrs=127.10.207.62:45045
--webserver_port=44075
--webserver_interface=127.10.207.2
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:41:03.956282 3213 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:41:03.956660 3213 tablet_bootstrap.cc:492] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Bootstrap complete.
I20260807 08:41:03.958125 3213 ts_tablet_manager.cc:1397] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.010s
I20260807 08:41:03.959623 3213 raft_consensus.cc:348] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:03.959952 3213 raft_consensus.cc:729] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9371bedbaa044a99fbd92052dd6630b, State: Initialized, Role: FOLLOWER
I20260807 08:41:03.960124 3213 consensus_queue.cc:260] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:03.960431 3213 ts_tablet_manager.cc:1428] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260807 08:41:04.019191 3377 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:03.950702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:52761)
0807 08:41:03.950962 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:03.951006 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:03.951561 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:04.018919 (+ 67358us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":152,"threads_started":1}
W20260807 08:41:04.019554 3370 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:45045 (0 consecutive failures): Network error: Failed to ping master at 127.10.207.62:45045: Client connection negotiation failed: client connection to 127.10.207.62:45045: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:41:04.020059 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.019917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:48977)
0807 08:41:04.019985 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.020038 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 08:41:04.020533 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.020432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:38231)
0807 08:41:04.020489 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.020516 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:41:04.020711 3370 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:41:04.053771 3379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:04.054003 3379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:04.054026 3379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:04.055282 3379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:04.055330 3379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:04.055357 3379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:41:04.056838 3379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:33335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=44075
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:04.057068 3379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:04.057282 3379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:04.062873 3385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:04.062902 3386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:04.062813 3388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:04.063055 3379 server_base.cc:1034] running on GCE node
I20260807 08:41:04.063371 3379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:04.063691 3379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:04.064888 3379 hybrid_clock.cc:648] HybridClock initialized: now 1786092064064864 us; error 60 us; skew 500 ppm
Aug 07 08:41:04 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:04.067701 3379 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:41:04.068150 3379 webserver.cc:460] Webserver started at http://127.10.207.2:44075/ using document root <none> and password file <none>
I20260807 08:41:04.068374 3379 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:04.068423 3379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:04.069782 3379 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:04.070429 3400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:04.070683 3379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:04.070770 3379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "7731efc76a9b4af6940a23e598c7a2af"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "34e1773f8ebf322e327b30edeb5427a1"
server_key_iv: "80b56acf964ae7a2bebb21e4a760b391"
server_key_version: "encryptionkey@0"
I20260807 08:41:04.071177 3379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:04.080075 3379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:04.082101 3379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:04.082252 3379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:04.082491 3379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:04.082850 3379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:04.082911 3379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:04.082947 3379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:04.082976 3379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:04.089859 3379 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:33335
I20260807 08:41:04.089972 3555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:33335 every 8 connection(s)
I20260807 08:41:04.090278 3379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:41:04.091501 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.091057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:55405)
0807 08:41:04.091245 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.091331 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
W20260807 08:41:04.091964 3556 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:45045 (0 consecutive failures): Network error: Failed to ping master at 127.10.207.62:45045: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
I20260807 08:41:04.092305 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.092188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:51751)
0807 08:41:04.092261 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.092289 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 08:41:04.092586 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.092529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:39671)
0807 08:41:04.092558 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.092575 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260807 08:41:04.092708 3556 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:41:04.095615 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 3379
I20260807 08:41:04.095947 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:34759
--local_ip_for_outbound_sockets=127.10.207.3
--tserver_master_addrs=127.10.207.62:45045
--webserver_port=35157
--webserver_interface=127.10.207.3
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:04.186203 3560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:04.186445 3560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:04.186486 3560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:04.187664 3560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:04.187736 3560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:04.187786 3560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:41:04.189483 3560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:34759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=35157
--tserver_master_addrs=127.10.207.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:04.189744 3560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:04.189977 3560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:04.195407 3568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:04.195421 3574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:04.195550 3569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:04.195868 3560 server_base.cc:1034] running on GCE node
I20260807 08:41:04.196161 3560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:04.196481 3560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:04.197686 3560 hybrid_clock.cc:648] HybridClock initialized: now 1786092064197669 us; error 48 us; skew 500 ppm
Aug 07 08:41:04 dist-test-slave-h6n0 krb5kdc[1322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:04.200389 3560 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:41:04.200870 3560 webserver.cc:460] Webserver started at http://127.10.207.3:35157/ using document root <none> and password file <none>
I20260807 08:41:04.201059 3560 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:04.201107 3560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:04.202387 3560 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:04.203172 3585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:04.203562 3560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:04.203656 3560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "e507b3fc1d7146e49f03c45f7c6976dc"
format_stamp: "Formatted at 2026-08-07 08:40:22 on dist-test-slave-h6n0"
server_key: "d6ca6beeabb254321550eeb63a9f126b"
server_key_iv: "1b76a43e440e947e315591c72c303799"
server_key_version: "encryptionkey@0"
I20260807 08:41:04.204140 3560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:04.220335 3560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:04.223161 3560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:04.223351 3560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:04.223624 3560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:04.223958 3560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:04.223996 3560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:04.224043 3560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:04.224076 3560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:04.231392 3560 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:34759
I20260807 08:41:04.231451 3766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:34759 every 8 connection(s)
I20260807 08:41:04.231781 3560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:41:04.232946 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 3560
I20260807 08:41:04.233076 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.232472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:57929)
0807 08:41:04.232656 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.232809 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
W20260807 08:41:04.233527 3768 heartbeater.cc:644] Failed to heartbeat to 127.10.207.62:45045 (0 consecutive failures): Network error: Failed to ping master at 127.10.207.62:45045: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
I20260807 08:41:04.233880 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.233745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:42897)
0807 08:41:04.233815 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.233846 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 08:41:04.234179 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:04.234080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:40979)
0807 08:41:04.234131 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:04.234154 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 08:41:04.234287 3768 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:41:05.021446 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:05.021160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:43481)
0807 08:41:05.021315 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:05.021361 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260807 08:41:05.093319 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:05.093058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:36475)
0807 08:41:05.093222 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:05.093265 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260807 08:41:05.107447 3780 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:05.107620 3780 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:05.108246 3780 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
Aug 07 08:41:05 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092063, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.2@KRBTEST.COM
Aug 07 08:41:05 dist-test-slave-h6n0 krb5kdc[1322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092063, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.3@KRBTEST.COM
I20260807 08:41:05.121778 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:05.108622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:52743 (local address 127.10.207.3:34759)
0807 08:41:05.108909 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:05.108921 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:05.108945 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:41:05.109526 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:05.109529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:05.109608 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:05.109789 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:05.110558 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.112202 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.113422 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.113956 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.114098 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:05.117545 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:05.117550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:05.117550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:05.117564 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:05.119970 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:05.120367 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:05.120369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:05.120370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:05.120458 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:05.120718 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:05.120719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:05.120719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:05.121021 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:05.121121 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:05.121489 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 08:41:05.121653 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":81,"threads_started":1}
I20260807 08:41:05.121793 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:05.108598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:43623 (local address 127.10.207.2:33335)
0807 08:41:05.108798 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:05.108810 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:05.108850 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:41:05.109286 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:05.109288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:05.109329 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:05.109454 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:05.110570 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.112201 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.113403 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.113962 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.114092 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:05.116839 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:05.116845 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:05.116845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:05.116860 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:05.119546 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:05.120057 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:05.120061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:05.120061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:05.120182 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:05.120525 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:05.120526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:05.120527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:05.120966 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:05.121078 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:05.121489 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 08:41:05.121659 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260807 08:41:05.121846 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:05.108509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:34759 (local address 127.10.207.1:52743)
0807 08:41:05.108663 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:05.108677 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:05.108753 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:05.109797 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:05.109798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:05.110099 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:05.110347 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:05.110353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.112366 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.112367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:05.113308 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:05.113311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.114068 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.114069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:05.114193 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:05.114875 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:05.114885 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:05.117387 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:05.120070 (+ 2683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:05.120071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:05.120071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:05.120261 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:05.120568 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:05.120569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:05.120569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:05.120627 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:05.121107 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:05.121108 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:05.121258 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 08:41:05.121396 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 08:41:05.121638 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":127,"thread_start_us":69,"threads_started":1}
I20260807 08:41:05.122267 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:05.108444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:33335 (local address 127.10.207.1:43623)
0807 08:41:05.108539 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:05.108551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:05.108610 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:05.109533 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:05.109538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:05.110091 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:05.110348 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:05.110355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.112316 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.112317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:05.113305 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:05.113307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:05.114066 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:05.114067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:05.114191 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:05.114945 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:05.114950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:05.116704 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:05.119673 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:05.119681 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:05.119683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:05.119952 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:05.120295 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:05.120296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:05.120297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:05.120436 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:05.121057 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:05.121084 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:05.121258 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 08:41:05.121395 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 08:41:05.121703 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":146}
W20260807 08:41:05.123131 3209 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
W20260807 08:41:05.123227 3208 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:41:05.123323 3208 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9371bedbaa044a99fbd92052dd6630b; no voters: 7731efc76a9b4af6940a23e598c7a2af, e507b3fc1d7146e49f03c45f7c6976dc
I20260807 08:41:05.123473 3780 raft_consensus.cc:2726] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:41:05.234987 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:05.234713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:53713)
0807 08:41:05.234864 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:05.234935 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260807 08:41:06.022365 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:06.022079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:40203)
0807 08:41:06.022249 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:06.022309 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260807 08:41:06.094163 3787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:06.093891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:52625)
0807 08:41:06.094067 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:06.094108 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260807 08:41:06.235841 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:06.235556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:53437)
0807 08:41:06.235735 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:06.235789 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260807 08:41:07.023263 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:07.022957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:34697)
0807 08:41:07.023147 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:07.023200 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260807 08:41:07.066895 3793 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:07.067030 3793 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:07.067256 3793 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
W20260807 08:41:07.067835 3209 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
W20260807 08:41:07.067916 3208 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:41:07.067970 3208 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9371bedbaa044a99fbd92052dd6630b; no voters: 7731efc76a9b4af6940a23e598c7a2af, e507b3fc1d7146e49f03c45f7c6976dc
I20260807 08:41:07.068061 3793 raft_consensus.cc:2726] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:41:07.094944 3795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:07.094679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:33287)
0807 08:41:07.094847 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:07.094890 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260807 08:41:07.236667 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:07.236425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:46285)
0807 08:41:07.236593 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:07.236633 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260807 08:41:08.024144 3797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:08.023869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:55183)
0807 08:41:08.024026 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:08.024079 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260807 08:41:08.095774 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:08.095477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:42789)
0807 08:41:08.095668 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:08.095721 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260807 08:41:08.237604 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:08.237334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:43749)
0807 08:41:08.237524 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:08.237564 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":89,"threads_started":1}
I20260807 08:41:09.025030 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:09.024765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:53883)
0807 08:41:09.024919 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:09.024968 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260807 08:41:09.096566 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:09.096305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:43531)
0807 08:41:09.096473 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:09.096516 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260807 08:41:09.238425 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:09.238158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:60807)
0807 08:41:09.238325 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:09.238370 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260807 08:41:09.900861 3807 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:09.901047 3807 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:09.901326 3807 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
W20260807 08:41:09.901813 3208 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
W20260807 08:41:09.902014 3209 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:41:09.902119 3209 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9371bedbaa044a99fbd92052dd6630b; no voters: 7731efc76a9b4af6940a23e598c7a2af, e507b3fc1d7146e49f03c45f7c6976dc
I20260807 08:41:09.902288 3807 raft_consensus.cc:2726] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:41:10.025985 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:10.025677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:35321)
0807 08:41:10.025853 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:10.025919 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260807 08:41:10.097376 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:10.097143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:48765)
0807 08:41:10.097287 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:10.097327 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260807 08:41:10.239288 3813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:10.239019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:36111)
0807 08:41:10.239170 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:10.239231 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260807 08:41:11.026978 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:11.026675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:48161)
0807 08:41:11.026861 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:11.026922 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":83,"threads_started":1}
I20260807 08:41:11.098290 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:11.097991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:40903)
0807 08:41:11.098176 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:11.098230 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":80,"threads_started":1}
I20260807 08:41:11.240154 3816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:11.239900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:50289)
0807 08:41:11.240062 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:11.240103 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260807 08:41:12.027984 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:12.027583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:33415)
0807 08:41:12.027858 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:12.027929 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":88,"threads_started":1}
I20260807 08:41:12.099267 3819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:12.098997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:52201)
0807 08:41:12.099161 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:12.099212 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260807 08:41:12.240978 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:12.240727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:46731)
0807 08:41:12.240892 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:12.240940 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260807 08:41:13.028841 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:13.028553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:54501)
0807 08:41:13.028723 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:13.028778 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":68,"threads_started":1}
I20260807 08:41:13.100129 3822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:13.099807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:54429)
0807 08:41:13.099979 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:13.100039 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260807 08:41:13.241828 3825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:13.241548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:51377)
0807 08:41:13.241716 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:13.241774 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":65,"threads_started":1}
I20260807 08:41:13.663689 3826 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:13.663828 3826 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:13.664212 3826 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
W20260807 08:41:13.664717 3208 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
W20260807 08:41:13.664814 3209 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:41:13.664956 3209 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9371bedbaa044a99fbd92052dd6630b; no voters: 7731efc76a9b4af6940a23e598c7a2af, e507b3fc1d7146e49f03c45f7c6976dc
I20260807 08:41:13.665081 3826 raft_consensus.cc:2726] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:41:14.029709 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:14.029404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:51025)
0807 08:41:14.029593 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:14.029656 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
I20260807 08:41:14.101032 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:14.100681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:36957)
0807 08:41:14.100885 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:14.100956 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
I20260807 08:41:14.242739 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:14.242430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:36069)
0807 08:41:14.242613 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:14.242685 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260807 08:41:15.030674 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:15.030349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:48129)
0807 08:41:15.030503 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:15.030596 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260807 08:41:15.101927 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:15.101628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:57173)
0807 08:41:15.101823 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:15.101881 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260807 08:41:15.243580 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:15.243299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:49541)
0807 08:41:15.243485 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:15.243531 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260807 08:41:15.791649 3836 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:15.791841 3836 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:15.792124 3836 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
W20260807 08:41:15.792692 3209 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
W20260807 08:41:15.792826 3208 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:41:15.792934 3208 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9371bedbaa044a99fbd92052dd6630b; no voters: 7731efc76a9b4af6940a23e598c7a2af, e507b3fc1d7146e49f03c45f7c6976dc
I20260807 08:41:15.793026 3836 raft_consensus.cc:2726] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:41:16.031494 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:16.031141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:55743)
0807 08:41:16.031370 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:16.031427 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
I20260807 08:41:16.102824 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:16.102491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:45863)
0807 08:41:16.102723 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:16.102766 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":155,"threads_started":1}
I20260807 08:41:16.244422 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:16.244141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:60835)
0807 08:41:16.244308 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:16.244364 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260807 08:41:17.032512 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:17.032223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:46075)
0807 08:41:17.032405 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:17.032458 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260807 08:41:17.103662 3844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:17.103393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:51079)
0807 08:41:17.103561 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:17.103599 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260807 08:41:17.245368 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:17.245083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:38043)
0807 08:41:17.245258 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:17.245305 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260807 08:41:17.668429 3848 raft_consensus.cc:482] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:17.668594 3848 raft_consensus.cc:504] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507b3fc1d7146e49f03c45f7c6976dc" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 34759 } } peers { permanent_uuid: "7731efc76a9b4af6940a23e598c7a2af" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 33335 } } peers { permanent_uuid: "e9371bedbaa044a99fbd92052dd6630b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39873 } }
I20260807 08:41:17.668849 3848 leader_election.cc:288] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759), 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335)
W20260807 08:41:17.669349 3209 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e507b3fc1d7146e49f03c45f7c6976dc (127.10.207.3:34759): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
W20260807 08:41:17.669494 3208 leader_election.cc:341] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7731efc76a9b4af6940a23e598c7a2af (127.10.207.2:33335): Not found: Tablet not found: 1e66ab9dcf0b4534b8d2ec2da31fcc39
I20260807 08:41:17.669559 3208 leader_election.cc:302] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9371bedbaa044a99fbd92052dd6630b; no voters: 7731efc76a9b4af6940a23e598c7a2af, e507b3fc1d7146e49f03c45f7c6976dc
I20260807 08:41:17.669642 3848 raft_consensus.cc:2726] T 1e66ab9dcf0b4534b8d2ec2da31fcc39 P e9371bedbaa044a99fbd92052dd6630b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:41:18.033339 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:18.033052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:60831)
0807 08:41:18.033232 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:18.033274 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260807 08:41:18.104506 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:18.104226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:59313)
0807 08:41:18.104389 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:18.104440 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260807 08:41:18.246321 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:18.245997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:50041)
0807 08:41:18.246222 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:18.246274 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":125,"threads_started":1}
I20260807 08:41:19.034356 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:19.034032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.1:40285)
0807 08:41:19.034251 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:19.034296 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":136,"threads_started":1}
I20260807 08:41:19.105422 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:19.105158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.2:50263)
0807 08:41:19.105322 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:19.105363 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260807 08:41:19.237265 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 3127
I20260807 08:41:19.247431 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:19.247139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:45045 (local address 127.10.207.3:48393)
0807 08:41:19.247322 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:19.247377 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.62:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260807 08:41:19.252640 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 3379
I20260807 08:41:19.264966 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 3560
2026-08-07T08:41:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58871 ms)
[----------] 30 tests from SecurityITest (249125 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:19 dist-test-slave-h6n0 krb5kdc[3861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:19 dist-test-slave-h6n0 krb5kdc[3861](info): set up 2 sockets
Aug 07 08:41:19 dist-test-slave-h6n0 krb5kdc[3861](info): commencing operation
krb5kdc: starting...
W20260807 08:41:20.543246 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:20 dist-test-slave-h6n0 krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:20.675657 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41679
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:41679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:20.767200 3885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:20.767436 3885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:20.767458 3885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:20.768703 3885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:20.768751 3885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:20.768765 3885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:20.768779 3885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:20.768816 3885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:20.770398 3885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:41679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:20.770707 3885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:20.770991 3885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:20.776849 3894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:20.776873 3885 server_base.cc:1034] running on GCE node
W20260807 08:41:20.776823 3891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:20.776829 3890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:20.777423 3885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:20.777745 3885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:20.778910 3885 hybrid_clock.cc:648] HybridClock initialized: now 1786092080778889 us; error 50 us; skew 500 ppm
Aug 07 08:41:20 dist-test-slave-h6n0 krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092080, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:20.781662 3885 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:20.782209 3885 webserver.cc:460] Webserver started at http://127.10.207.62:38759/ using document root <none> and password file <none>
I20260807 08:41:20.782452 3885 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:20.782521 3885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:20.782706 3885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:20.783865 3885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "e534b48f4f734781a209027fd8c0dc8e"
format_stamp: "Formatted at 2026-08-07 08:41:20 on dist-test-slave-h6n0"
server_key: "80ac58d6e99c2a13067886750e428d68"
server_key_iv: "03f3b3700e117befc53593976c3c74eb"
server_key_version: "encryptionkey@0"
I20260807 08:41:20.784271 3885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "e534b48f4f734781a209027fd8c0dc8e"
format_stamp: "Formatted at 2026-08-07 08:41:20 on dist-test-slave-h6n0"
server_key: "80ac58d6e99c2a13067886750e428d68"
server_key_iv: "03f3b3700e117befc53593976c3c74eb"
server_key_version: "encryptionkey@0"
I20260807 08:41:20.785785 3885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:20.786689 3902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:20.786916 3885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:20.787035 3885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "e534b48f4f734781a209027fd8c0dc8e"
format_stamp: "Formatted at 2026-08-07 08:41:20 on dist-test-slave-h6n0"
server_key: "80ac58d6e99c2a13067886750e428d68"
server_key_iv: "03f3b3700e117befc53593976c3c74eb"
server_key_version: "encryptionkey@0"
I20260807 08:41:20.787132 3885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:20.795956 3885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:20.798029 3885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:20.798198 3885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:20.803442 3885 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:41679
I20260807 08:41:20.803483 3980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:41679 every 8 connection(s)
I20260807 08:41:20.803891 3885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:20.805022 3981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:20.807837 3981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e: Bootstrap starting.
I20260807 08:41:20.808457 3981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:20.808777 3981 log.cc:824] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:20.809619 3981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e: No bootstrap required, opened a new log
I20260807 08:41:20.811336 3981 raft_consensus.cc:348] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e534b48f4f734781a209027fd8c0dc8e" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41679 } }
I20260807 08:41:20.811547 3981 raft_consensus.cc:374] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:20.811547 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 3885
I20260807 08:41:20.811594 3981 raft_consensus.cc:729] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e534b48f4f734781a209027fd8c0dc8e, State: Initialized, Role: FOLLOWER
I20260807 08:41:20.811646 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:20.811774 3981 consensus_queue.cc:260] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e534b48f4f734781a209027fd8c0dc8e" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41679 } }
I20260807 08:41:20.811852 11068 external_mini_cluster.cc:1428] Setting key aa8672fcc3b600392c52ac5f2468a742
I20260807 08:41:20.811868 3981 raft_consensus.cc:388] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:20.811906 3981 raft_consensus.cc:482] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:20.811967 3981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:20.812587 3981 raft_consensus.cc:504] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e534b48f4f734781a209027fd8c0dc8e" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41679 } }
I20260807 08:41:20.812719 3981 leader_election.cc:302] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e534b48f4f734781a209027fd8c0dc8e; no voters:
I20260807 08:41:20.812911 3981 leader_election.cc:288] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:20.812978 3990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:20.813184 3981 sys_catalog.cc:564] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:20.813272 3990 raft_consensus.cc:686] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [term 1 LEADER]: Becoming Leader. State: Replica: e534b48f4f734781a209027fd8c0dc8e, State: Running, Role: LEADER
I20260807 08:41:20.813416 3990 consensus_queue.cc:237] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e534b48f4f734781a209027fd8c0dc8e" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41679 } }
I20260807 08:41:20.813897 3990 sys_catalog.cc:455] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e534b48f4f734781a209027fd8c0dc8e. Latest consensus state: current_term: 1 leader_uuid: "e534b48f4f734781a209027fd8c0dc8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e534b48f4f734781a209027fd8c0dc8e" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41679 } } }
I20260807 08:41:20.813988 3990 sys_catalog.cc:458] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:20.814085 3992 sys_catalog.cc:455] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e534b48f4f734781a209027fd8c0dc8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e534b48f4f734781a209027fd8c0dc8e" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41679 } } }
I20260807 08:41:20.814175 3992 sys_catalog.cc:458] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:20.815469 4010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:20.815534 4010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:20.815990 4002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:20.816226 4002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:20.818069 4002 catalog_manager.cc:1347] Generated new cluster ID: d5cd795fa562487b8f6cef07478958fa
I20260807 08:41:20.818143 4002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:41:20 dist-test-slave-h6n0 krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:20.824484 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:20.812443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.10.207.62:41679)
0807 08:41:20.812670 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:20.812685 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:20.812729 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:41:20.812913 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:20.812924 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:20.812998 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:20.814086 (+ 1088us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:20.814239 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:20.815840 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:20.817016 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:20.817664 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:20.817792 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:20.820493 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:20.820499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:20.820499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:20.820513 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:20.822829 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:20.823215 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:20.823219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:20.823219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:20.823333 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:20.823546 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:20.823547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:20.823548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:20.823811 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:20.824020 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:20.824103 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 08:41:20.824273 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
I20260807 08:41:20.831112 4002 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:20.831875 4002 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:20.838685 4002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e: Generated new TSK 0
I20260807 08:41:20.838886 4002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:41:20.870998 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41679
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:20.970124 4018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:20.970389 4018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:20.970417 4018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:20.971750 4018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:20.971808 4018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:20.971840 4018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:41:20.973404 4018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:20.973657 4018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:20.973884 4018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:20.979656 4027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:20.979676 4032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:20.979676 4029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:20.980039 4018 server_base.cc:1034] running on GCE node
I20260807 08:41:20.980289 4018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:20.980574 4018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:20.981727 4018 hybrid_clock.cc:648] HybridClock initialized: now 1786092080981693 us; error 54 us; skew 500 ppm
Aug 07 08:41:20 dist-test-slave-h6n0 krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092080, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:20.984664 4018 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:41:20.985176 4018 webserver.cc:460] Webserver started at http://127.10.207.1:39927/ using document root <none> and password file <none>
I20260807 08:41:20.985404 4018 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:20.985450 4018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:20.985549 4018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:20.986838 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "3af62bd26f3442a198ca64bdc0f5b7f5"
format_stamp: "Formatted at 2026-08-07 08:41:20 on dist-test-slave-h6n0"
server_key: "ee02b3aadebc9fa08b0f59ddd4c736fc"
server_key_iv: "fe9ab55b06a63428081a029e97ff6a1d"
server_key_version: "encryptionkey@0"
I20260807 08:41:20.987200 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "3af62bd26f3442a198ca64bdc0f5b7f5"
format_stamp: "Formatted at 2026-08-07 08:41:20 on dist-test-slave-h6n0"
server_key: "ee02b3aadebc9fa08b0f59ddd4c736fc"
server_key_iv: "fe9ab55b06a63428081a029e97ff6a1d"
server_key_version: "encryptionkey@0"
I20260807 08:41:20.988718 4018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:20.989595 4041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:20.989830 4018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:20.989943 4018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "3af62bd26f3442a198ca64bdc0f5b7f5"
format_stamp: "Formatted at 2026-08-07 08:41:20 on dist-test-slave-h6n0"
server_key: "ee02b3aadebc9fa08b0f59ddd4c736fc"
server_key_iv: "fe9ab55b06a63428081a029e97ff6a1d"
server_key_version: "encryptionkey@0"
I20260807 08:41:20.990029 4018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:21.009063 4018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:21.011737 4018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:21.011914 4018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:21.012235 4018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:21.012627 4018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:21.012683 4018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.012739 4018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:21.012780 4018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.021166 4018 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:36565
I20260807 08:41:21.021219 4214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:36565 every 8 connection(s)
I20260807 08:41:21.021622 4018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:41:21 dist-test-slave-h6n0 krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092080, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:21.028908 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 4018
I20260807 08:41:21.029045 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:41:21.029251 11068 external_mini_cluster.cc:1428] Setting key c4289980f496b58aa12573f7feed1cd6
I20260807 08:41:21.033795 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.022682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:56791 (local address 127.10.207.62:41679)
0807 08:41:21.022759 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.022762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.022949 (+ 187us) server_negotiation.cc:407] Connection header received
0807 08:41:21.023518 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.023519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.023531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.023625 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:21.024723 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.025298 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.026391 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.026638 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.026708 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.029569 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:21.029572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:21.029572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:21.029598 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:21.032014 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.032543 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.032545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.032545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.032615 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.033019 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.033020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.033020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.033144 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:21.033210 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.033636 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.033725 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:41:21.034125 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.022594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41679 (local address 127.10.207.1:56791)
0807 08:41:21.022814 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.022861 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.023412 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.023717 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.023721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.024374 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:21.024596 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.024600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.025409 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.025410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.026293 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.026295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.026726 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.026727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.026831 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.027453 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:21.027462 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:21.029419 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:21.032151 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:21.032161 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:21.032164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:21.032452 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:21.032721 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:21.032722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:21.032722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:21.032923 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:21.033225 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:21.033233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:21.033442 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.033588 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.033819 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":123,"threads_started":1}
I20260807 08:41:21.034861 4216 heartbeater.cc:347] Connected to a master server at 127.10.207.62:41679
I20260807 08:41:21.034996 4216 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:21.035210 4216 heartbeater.cc:511] Master 127.10.207.62:41679 requested a full tablet report, sending...
I20260807 08:41:21.035997 3930 ts_manager.cc:194] Registered new tserver with Master: 3af62bd26f3442a198ca64bdc0f5b7f5 (127.10.207.1:36565)
I20260807 08:41:21.037050 3930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:56791
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:41:21.061467 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41679
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:21.155037 4222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:21.155319 4222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:21.155357 4222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:21.156504 4222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:21.156569 4222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:21.156615 4222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:41:21.158169 4222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:21.158481 4222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:21.158766 4222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:21.164362 4228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:21.164391 4232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:21.164391 4234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:21.164743 4222 server_base.cc:1034] running on GCE node
I20260807 08:41:21.164985 4222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:21.165238 4222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:21.166414 4222 hybrid_clock.cc:648] HybridClock initialized: now 1786092081166388 us; error 52 us; skew 500 ppm
Aug 07 08:41:21 dist-test-slave-h6n0 krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092081, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:21.169298 4222 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:41:21.169821 4222 webserver.cc:460] Webserver started at http://127.10.207.2:32837/ using document root <none> and password file <none>
I20260807 08:41:21.170089 4222 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:21.170157 4222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:21.170316 4222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:21.171505 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "94293bf5861f4fe39164fb9c2bac2300"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "9657908c8877be3d97b5b13beac060e6"
server_key_iv: "51887c597ca5db81e218fb1c9d634981"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.171892 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "94293bf5861f4fe39164fb9c2bac2300"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "9657908c8877be3d97b5b13beac060e6"
server_key_iv: "51887c597ca5db81e218fb1c9d634981"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.173442 4222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:21.174466 4243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.174759 4222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:21.174875 4222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "94293bf5861f4fe39164fb9c2bac2300"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "9657908c8877be3d97b5b13beac060e6"
server_key_iv: "51887c597ca5db81e218fb1c9d634981"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.174961 4222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:21.192853 4222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:21.195472 4222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:21.195655 4222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:21.195931 4222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:21.196308 4222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:21.196362 4222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.196400 4222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:21.196444 4222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.204305 4222 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:43187
I20260807 08:41:21.204360 4412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:43187 every 8 connection(s)
I20260807 08:41:21.204749 4222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:41:21.207651 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 4222
I20260807 08:41:21.207803 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:41:21.208062 11068 external_mini_cluster.cc:1428] Setting key bc7dbaa6a25d9417bd9f9b11c0ea4acc
Aug 07 08:41:21 dist-test-slave-h6n0 krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092081, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:21.217027 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.205701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:49087 (local address 127.10.207.62:41679)
0807 08:41:21.205780 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.205782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.205981 (+ 199us) server_negotiation.cc:407] Connection header received
0807 08:41:21.206629 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.206631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.206647 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.206724 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:21.207842 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.208619 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.209818 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.210043 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.210121 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.213030 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:21.213032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:21.213032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:21.213041 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:21.214872 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.215443 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.215445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.215446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.215561 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.215953 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.215955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.215955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.216147 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:21.216227 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.216649 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.216839 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:41:21.217303 4416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.205644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41679 (local address 127.10.207.2:49087)
0807 08:41:21.205831 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.205879 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.206469 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.206793 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.206797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.207446 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:21.207685 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.207690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.208774 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.208775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.209706 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.209709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.210131 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.210133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.210281 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.210979 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:21.210989 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:21.212927 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:21.214984 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:21.214992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:21.214995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:21.215333 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:21.215663 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:21.215664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:21.215665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:21.215852 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:21.216244 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:21.216254 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:21.216458 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.216598 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.216871 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260807 08:41:21.218516 4414 heartbeater.cc:347] Connected to a master server at 127.10.207.62:41679
I20260807 08:41:21.218675 4414 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:21.218923 4414 heartbeater.cc:511] Master 127.10.207.62:41679 requested a full tablet report, sending...
I20260807 08:41:21.219404 3930 ts_manager.cc:194] Registered new tserver with Master: 94293bf5861f4fe39164fb9c2bac2300 (127.10.207.2:43187)
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
I20260807 08:41:21.219970 3930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:49087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:41:21.238479 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41679
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:21.325178 4419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:21.325373 4419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:21.325397 4419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:21.326450 4419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:21.326491 4419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:21.326573 4419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:41:21.327987 4419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:41679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:21.328174 4419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:21.328354 4419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:21.333371 4433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:21.333505 4419 server_base.cc:1034] running on GCE node
W20260807 08:41:21.333364 4428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:21.333611 4427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:21.333917 4419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:21.334162 4419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:21.335340 4419 hybrid_clock.cc:648] HybridClock initialized: now 1786092081335315 us; error 44 us; skew 500 ppm
Aug 07 08:41:21 dist-test-slave-h6n0 krb5kdc[3861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092081, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:21.338297 4419 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:41:21.338943 4419 webserver.cc:460] Webserver started at http://127.10.207.3:36715/ using document root <none> and password file <none>
I20260807 08:41:21.339251 4419 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:21.339316 4419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:21.339495 4419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:21.340692 4419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "057e5b435f904826b89b6a9ed15c54f9"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "7cb0fcb89358fe19187af82a5de04033"
server_key_iv: "65a3f37a1468956699a0b498606de41f"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.341125 4419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "057e5b435f904826b89b6a9ed15c54f9"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "7cb0fcb89358fe19187af82a5de04033"
server_key_iv: "65a3f37a1468956699a0b498606de41f"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.342669 4419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:21.343565 4444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.343752 4419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:21.343858 4419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "057e5b435f904826b89b6a9ed15c54f9"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "7cb0fcb89358fe19187af82a5de04033"
server_key_iv: "65a3f37a1468956699a0b498606de41f"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.343953 4419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:21.361362 4419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:21.363472 4419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:21.363600 4419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:21.363804 4419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:21.364115 4419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:21.364154 4419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.364178 4419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:21.364193 4419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.371451 4419 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:36655
I20260807 08:41:21.371510 4621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:36655 every 8 connection(s)
I20260807 08:41:21.371814 4419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:41:21.375032 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 4419
I20260807 08:41:21.375149 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:41:21.375366 11068 external_mini_cluster.cc:1428] Setting key 569ad692b972d4333250d20077ca6a19
Aug 07 08:41:21 dist-test-slave-h6n0 krb5kdc[3861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092081, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:21.382501 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.372709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:59213 (local address 127.10.207.62:41679)
0807 08:41:21.372813 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.372816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.372959 (+ 143us) server_negotiation.cc:407] Connection header received
0807 08:41:21.373591 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.373592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.373606 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.373683 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:21.374820 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.375531 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.376992 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.377215 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.377288 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.379916 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:21.379918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:21.379918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:21.379926 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:21.381216 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.381616 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.381616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.381617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.381678 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.381913 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.381914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.381914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.382024 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:21.382089 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.382364 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.382435 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:41:21.382771 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.372630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41679 (local address 127.10.207.3:59213)
0807 08:41:21.372806 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.372848 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.373441 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.373766 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.373770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.374418 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:21.374714 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.374719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.375687 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.375689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.376902 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.376905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.377304 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.377305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.377412 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.378047 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:21.378058 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:21.379810 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:21.381306 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:21.381312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:21.381314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:21.381538 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:21.381747 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:21.381748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:21.381748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:21.381849 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:21.382115 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:21.382120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:21.382237 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.382327 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.382456 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260807 08:41:21.383296 4622 heartbeater.cc:347] Connected to a master server at 127.10.207.62:41679
I20260807 08:41:21.383404 4622 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:21.383601 4622 heartbeater.cc:511] Master 127.10.207.62:41679 requested a full tablet report, sending...
I20260807 08:41:21.384024 3930 ts_manager.cc:194] Registered new tserver with Master: 057e5b435f904826b89b6a9ed15c54f9 (127.10.207.3:36655)
I20260807 08:41:21.384438 3930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:59213
I20260807 08:41:21.387696 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:41:21.396406 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.389381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.10.207.62:41679)
0807 08:41:21.389474 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.389477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.389635 (+ 158us) server_negotiation.cc:407] Connection header received
0807 08:41:21.389733 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.389734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.389748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.389851 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:21.390408 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.391021 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.391787 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.391992 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.392052 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.393507 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:21.393509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:21.393509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:21.393516 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:21.395120 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.395505 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.395506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.395506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.395562 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:21.395779 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:21.395780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:21.395780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:21.395937 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:21.396059 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.396190 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.396277 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:41:21.398177 3930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:41:21.400239 3930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:41:21.406984 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.402857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36565 (local address 127.0.0.1:37746)
0807 08:41:21.403056 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.403109 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.403186 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.403509 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.403513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.403517 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:21.403589 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.403591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.404561 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.404562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.405577 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.405580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.406660 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.406661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.406744 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.406758 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.406845 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.406895 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":63,"threads_started":1}
I20260807 08:41:21.407027 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.403033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.10.207.1:36565)
0807 08:41:21.403188 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.403194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.403209 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:41:21.403295 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.403297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.403314 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.403411 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:21.403680 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.404439 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.405669 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.406561 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.406646 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.406713 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.406877 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.406929 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260807 08:41:21.407639 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.403098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43187 (local address 127.0.0.1:37858)
0807 08:41:21.403342 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.403352 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.403396 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.403766 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.403767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.403768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:21.403817 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.403818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.405185 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.405186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.406120 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.406122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.407423 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.407424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.407482 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.407487 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.407550 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.407594 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":79,"threads_started":1}
I20260807 08:41:21.407953 4659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.403343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.10.207.2:43187)
0807 08:41:21.403559 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.403588 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.403597 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:41:21.403643 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.403645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.403660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.403892 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:21.403972 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.405062 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.406205 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.406990 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.407390 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.407451 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.407781 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.407868 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260807 08:41:21.408097 4120 tablet_service.cc:1467] Processing CreateTablet for tablet bf6cd24e31d342adae58dfbf4f3bbdab (DEFAULT_TABLE table=test-table [id=e2fe22cb56d543fa9cb3b130a5ab1214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:21.408608 4647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.402934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.10.207.3:36655)
0807 08:41:21.403217 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.403221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.403467 (+ 246us) server_negotiation.cc:407] Connection header received
0807 08:41:21.403521 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.403524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.403542 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.403622 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:21.403845 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.405015 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.407238 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.407973 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.408062 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.408126 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.408453 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.408508 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
I20260807 08:41:21.408833 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.402857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:36655 (local address 127.0.0.1:39842)
0807 08:41:21.403312 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.403323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.403395 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.403703 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.403705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.403708 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:21.403767 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.403769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.406209 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.406209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.407156 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.407159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.408173 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.408174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.408268 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.408273 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.408329 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.408756 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":20,"thread_start_us":100,"threads_started":1}
I20260807 08:41:21.408824 4120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6cd24e31d342adae58dfbf4f3bbdab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:21.409133 4319 tablet_service.cc:1467] Processing CreateTablet for tablet bf6cd24e31d342adae58dfbf4f3bbdab (DEFAULT_TABLE table=test-table [id=e2fe22cb56d543fa9cb3b130a5ab1214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:21.409483 4319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6cd24e31d342adae58dfbf4f3bbdab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:21.411936 4661 tablet_bootstrap.cc:492] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: Bootstrap starting.
I20260807 08:41:21.412328 4663 tablet_bootstrap.cc:492] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Bootstrap starting.
I20260807 08:41:21.412679 4661 tablet_bootstrap.cc:654] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:21.412727 4524 tablet_service.cc:1467] Processing CreateTablet for tablet bf6cd24e31d342adae58dfbf4f3bbdab (DEFAULT_TABLE table=test-table [id=e2fe22cb56d543fa9cb3b130a5ab1214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:21.413045 4663 tablet_bootstrap.cc:654] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:21.413051 4524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6cd24e31d342adae58dfbf4f3bbdab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:21.413309 4661 log.cc:824] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:21.413650 4663 log.cc:824] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:21.414146 4661 tablet_bootstrap.cc:492] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: No bootstrap required, opened a new log
I20260807 08:41:21.414212 4661 ts_tablet_manager.cc:1397] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:41:21.415216 4663 tablet_bootstrap.cc:492] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: No bootstrap required, opened a new log
I20260807 08:41:21.415297 4663 ts_tablet_manager.cc:1397] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:41:21.416162 4661 raft_consensus.cc:348] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.416404 4661 raft_consensus.cc:374] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:21.416456 4661 raft_consensus.cc:729] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3af62bd26f3442a198ca64bdc0f5b7f5, State: Initialized, Role: FOLLOWER
I20260807 08:41:21.416373 4670 tablet_bootstrap.cc:492] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Bootstrap starting.
I20260807 08:41:21.416622 4661 consensus_queue.cc:260] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.416955 4661 ts_tablet_manager.cc:1428] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:41:21.417124 4216 heartbeater.cc:503] Master 127.10.207.62:41679 was elected leader, sending a full tablet report...
I20260807 08:41:21.417389 4670 tablet_bootstrap.cc:654] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:21.417302 4663 raft_consensus.cc:348] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.417476 4663 raft_consensus.cc:374] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:21.417519 4663 raft_consensus.cc:729] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94293bf5861f4fe39164fb9c2bac2300, State: Initialized, Role: FOLLOWER
I20260807 08:41:21.417672 4663 consensus_queue.cc:260] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.417970 4663 ts_tablet_manager.cc:1428] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:41:21.418033 4670 log.cc:824] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:21.418116 4414 heartbeater.cc:503] Master 127.10.207.62:41679 was elected leader, sending a full tablet report...
I20260807 08:41:21.418958 4670 tablet_bootstrap.cc:492] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: No bootstrap required, opened a new log
I20260807 08:41:21.419037 4670 ts_tablet_manager.cc:1397] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:41:21.420516 4670 raft_consensus.cc:348] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.420660 4670 raft_consensus.cc:374] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:21.420683 4670 raft_consensus.cc:729] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057e5b435f904826b89b6a9ed15c54f9, State: Initialized, Role: FOLLOWER
I20260807 08:41:21.420817 4670 consensus_queue.cc:260] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.421097 4670 ts_tablet_manager.cc:1428] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:41:21.421164 4622 heartbeater.cc:503] Master 127.10.207.62:41679 was elected leader, sending a full tablet report...
W20260807 08:41:21.455430 4415 tablet.cc:2369] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:41:21.522599 4217 tablet.cc:2369] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:41:21.542461 4674 raft_consensus.cc:482] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:21.542657 4674 raft_consensus.cc:504] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.543052 4674 leader_election.cc:288] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3af62bd26f3442a198ca64bdc0f5b7f5 (127.10.207.1:36565), 94293bf5861f4fe39164fb9c2bac2300 (127.10.207.2:43187)
I20260807 08:41:21.546371 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.543415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:57415 (local address 127.10.207.1:36565)
0807 08:41:21.543477 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.543480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.543493 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:41:21.543633 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.543635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.543651 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.543720 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:21.543961 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.544495 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.545600 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.545967 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.546039 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.546046 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.546245 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.546289 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:41:21.546391 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.543276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36565 (local address 127.10.207.3:57415)
0807 08:41:21.543376 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.543395 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.543494 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.543742 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.543744 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.543746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:21.543864 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.543867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.544579 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.544581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.545493 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.545496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.546050 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.546051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.546157 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.546162 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.546220 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.546276 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":5}
I20260807 08:41:21.546481 4659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.543486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:56477 (local address 127.10.207.2:43187)
0807 08:41:21.543578 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.543580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.543708 (+ 128us) server_negotiation.cc:407] Connection header received
0807 08:41:21.543753 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.543754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.543771 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.543845 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:21.544054 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.544584 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.545636 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.546070 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.546154 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.546166 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.546352 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.546398 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:41:21.546629 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.543396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:43187 (local address 127.10.207.3:56477)
0807 08:41:21.543603 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.543616 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:21.543694 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:21.543859 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:21.543860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:21.543861 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:21.543970 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.543972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.544693 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.544694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.545512 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:21.545514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.546166 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.546167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:21.546263 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.546268 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:21.546334 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:41:21.546381 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":1,"thread_start_us":129,"threads_started":1}
I20260807 08:41:21.547034 4345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6cd24e31d342adae58dfbf4f3bbdab" candidate_uuid: "057e5b435f904826b89b6a9ed15c54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94293bf5861f4fe39164fb9c2bac2300" is_pre_election: true
I20260807 08:41:21.547067 4144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6cd24e31d342adae58dfbf4f3bbdab" candidate_uuid: "057e5b435f904826b89b6a9ed15c54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" is_pre_election: true
I20260807 08:41:21.547207 4345 raft_consensus.cc:2454] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 057e5b435f904826b89b6a9ed15c54f9 in term 0.
I20260807 08:41:21.547289 4144 raft_consensus.cc:2454] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 057e5b435f904826b89b6a9ed15c54f9 in term 0.
I20260807 08:41:21.547513 4452 leader_election.cc:302] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 057e5b435f904826b89b6a9ed15c54f9, 94293bf5861f4fe39164fb9c2bac2300; no voters:
I20260807 08:41:21.547675 4674 raft_consensus.cc:2781] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:41:21.547757 4674 raft_consensus.cc:482] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:41:21.547797 4674 raft_consensus.cc:3037] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:21.548568 4674 raft_consensus.cc:504] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.548713 4674 leader_election.cc:288] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [CANDIDATE]: Term 1 election: Requested vote from peers 3af62bd26f3442a198ca64bdc0f5b7f5 (127.10.207.1:36565), 94293bf5861f4fe39164fb9c2bac2300 (127.10.207.2:43187)
I20260807 08:41:21.548936 4345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6cd24e31d342adae58dfbf4f3bbdab" candidate_uuid: "057e5b435f904826b89b6a9ed15c54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94293bf5861f4fe39164fb9c2bac2300"
I20260807 08:41:21.548960 4144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6cd24e31d342adae58dfbf4f3bbdab" candidate_uuid: "057e5b435f904826b89b6a9ed15c54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5"
I20260807 08:41:21.549031 4345 raft_consensus.cc:3037] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:21.549044 4144 raft_consensus.cc:3037] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:21.549749 4345 raft_consensus.cc:2454] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 057e5b435f904826b89b6a9ed15c54f9 in term 1.
I20260807 08:41:21.550028 4452 leader_election.cc:302] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 057e5b435f904826b89b6a9ed15c54f9, 94293bf5861f4fe39164fb9c2bac2300; no voters:
I20260807 08:41:21.550158 4144 raft_consensus.cc:2454] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 057e5b435f904826b89b6a9ed15c54f9 in term 1.
I20260807 08:41:21.550402 4674 raft_consensus.cc:2781] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:21.550631 4674 raft_consensus.cc:686] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 1 LEADER]: Becoming Leader. State: Replica: 057e5b435f904826b89b6a9ed15c54f9, State: Running, Role: LEADER
I20260807 08:41:21.550765 4674 consensus_queue.cc:237] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } }
I20260807 08:41:21.551878 3925 catalog_manager.cc:5495] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 057e5b435f904826b89b6a9ed15c54f9 (127.10.207.3). New cstate: current_term: 1 leader_uuid: "057e5b435f904826b89b6a9ed15c54f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "057e5b435f904826b89b6a9ed15c54f9" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260807 08:41:21.593947 4647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.591229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.10.207.3:36655)
0807 08:41:21.591287 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.591289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.591332 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:41:21.591434 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.591434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.591460 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.591533 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:41:21.591660 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.592205 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.593003 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.593234 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.593304 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.593444 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:41:21.593618 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:41:21.593712 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.593825 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.593848 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:41:21.596357 4144 raft_consensus.cc:1261] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 1 FOLLOWER]: Refusing update from remote peer 057e5b435f904826b89b6a9ed15c54f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:21.596391 4345 raft_consensus.cc:1261] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 1 FOLLOWER]: Refusing update from remote peer 057e5b435f904826b89b6a9ed15c54f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:21.596801 4678 consensus_queue.cc:1035] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3af62bd26f3442a198ca64bdc0f5b7f5" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:21.596968 4674 consensus_queue.cc:1035] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94293bf5861f4fe39164fb9c2bac2300" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 43187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:21.600313 4681 mvcc.cc:205] Tried to move back new op lower bound from 7315833166215389184 to 7315833166032809984. Current Snapshot: MvccSnapshot[applied={T|T < 7315833166215389184}]
I20260807 08:41:21.600610 4682 mvcc.cc:205] Tried to move back new op lower bound from 7315833166215389184 to 7315833166032809984. Current Snapshot: MvccSnapshot[applied={T|T < 7315833166215389184}]
I20260807 08:41:21.603072 3927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:21.603212 3927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:21.604292 3927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e534b48f4f734781a209027fd8c0dc8e: Sending DeleteTablet for 3 replicas of tablet bf6cd24e31d342adae58dfbf4f3bbdab
I20260807 08:41:21.604732 4120 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6cd24e31d342adae58dfbf4f3bbdab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:21 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37746
I20260807 08:41:21.604753 4319 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6cd24e31d342adae58dfbf4f3bbdab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:21 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:37858
I20260807 08:41:21.605054 4524 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6cd24e31d342adae58dfbf4f3bbdab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:21 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:39842
I20260807 08:41:21.605202 4692 tablet_replica.cc:330] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5: stopping tablet replica
I20260807 08:41:21.605329 4692 raft_consensus.cc:2229] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:21.605356 4694 tablet_replica.cc:330] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: stopping tablet replica
I20260807 08:41:21.605455 4692 raft_consensus.cc:2258] T bf6cd24e31d342adae58dfbf4f3bbdab P 3af62bd26f3442a198ca64bdc0f5b7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:21.605471 4694 raft_consensus.cc:2229] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:21.605477 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 4018
I20260807 08:41:21.620666 4694 raft_consensus.cc:2258] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:21.622426 4694 ts_tablet_manager.cc:1905] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:21.625056 4694 ts_tablet_manager.cc:1918] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:21.625154 4694 log.cc:1196] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/bf6cd24e31d342adae58dfbf4f3bbdab
I20260807 08:41:21.625589 4694 ts_tablet_manager.cc:1939] T bf6cd24e31d342adae58dfbf4f3bbdab P 94293bf5861f4fe39164fb9c2bac2300: Deleting consensus metadata
I20260807 08:41:21.626211 3907 catalog_manager.cc:4841] TS 94293bf5861f4fe39164fb9c2bac2300 (127.10.207.2:43187): tablet bf6cd24e31d342adae58dfbf4f3bbdab (table test-table [id=e2fe22cb56d543fa9cb3b130a5ab1214]) successfully deleted
I20260807 08:41:21.630626 4695 tablet_replica.cc:330] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: stopping tablet replica
I20260807 08:41:21.630821 4695 raft_consensus.cc:2229] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:41:21.630982 4695 raft_consensus.cc:2258] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:21.632201 4695 ts_tablet_manager.cc:1905] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:21.633747 4695 ts_tablet_manager.cc:1918] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:21.633846 4695 log.cc:1196] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/bf6cd24e31d342adae58dfbf4f3bbdab
I20260807 08:41:21.634135 4695 ts_tablet_manager.cc:1939] T bf6cd24e31d342adae58dfbf4f3bbdab P 057e5b435f904826b89b6a9ed15c54f9: Deleting consensus metadata
I20260807 08:41:21.634651 3905 catalog_manager.cc:4841] TS 057e5b435f904826b89b6a9ed15c54f9 (127.10.207.3:36655): tablet bf6cd24e31d342adae58dfbf4f3bbdab (table test-table [id=e2fe22cb56d543fa9cb3b130a5ab1214]) successfully deleted
W20260807 08:41:21.638978 3907 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:36565 (error 108)
I20260807 08:41:21.639626 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.639499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:36565 (local address 127.0.0.1:37748)
0807 08:41:21.639561 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:21.639597 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 08:41:21.639778 3907 catalog_manager.cc:4584] TS 3af62bd26f3442a198ca64bdc0f5b7f5 (127.10.207.1:36565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf6cd24e31d342adae58dfbf4f3bbdab: Network error: Client connection negotiation failed: client connection to 127.10.207.1:36565: connect: Connection refused (error 111)
I20260807 08:41:21.639870 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 4222
I20260807 08:41:21.666501 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 4419
I20260807 08:41:21.678789 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 3885
2026-08-07T08:41:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2416 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T08:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:21Z Disabled control of system clock
I20260807 08:41:21.719661 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:34979
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:38647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:34979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:41:21.810463 4703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:21.810787 4703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:21.810811 4703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:21.812000 4703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:21.812043 4703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:21.812057 4703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:21.812070 4703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:21.812083 4703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:21.813419 4703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:34979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:34979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:21.813643 4703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:21.813928 4703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:21.819471 4711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:21.819459 4712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:21.819458 4714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:21.819854 4703 server_base.cc:1034] running on GCE node
I20260807 08:41:21.820147 4703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:21.820474 4703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:21.821642 4703 hybrid_clock.cc:648] HybridClock initialized: now 1786092081821630 us; error 44 us; skew 500 ppm
I20260807 08:41:21.823114 4703 webserver.cc:460] Webserver started at http://127.10.207.62:38757/ using document root <none> and password file <none>
I20260807 08:41:21.823376 4703 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:21.823447 4703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:21.823598 4703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:21.824815 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "949ad7c3487b4f5d88b71e02ccc4979d"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "50a31e6c7af142da14ebb753b4caa1ef"
server_key_iv: "2206f589d86f0787920f45d9e57bf624"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.825202 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "949ad7c3487b4f5d88b71e02ccc4979d"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "50a31e6c7af142da14ebb753b4caa1ef"
server_key_iv: "2206f589d86f0787920f45d9e57bf624"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.826781 4703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:41:21.827598 4724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:21.827809 4703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:21.827932 4703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "949ad7c3487b4f5d88b71e02ccc4979d"
format_stamp: "Formatted at 2026-08-07 08:41:21 on dist-test-slave-h6n0"
server_key: "50a31e6c7af142da14ebb753b4caa1ef"
server_key_iv: "2206f589d86f0787920f45d9e57bf624"
server_key_version: "encryptionkey@0"
I20260807 08:41:21.828011 4703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:21.838951 4703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:21.839236 4703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:21.839385 4703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:21.843753 4703 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:34979
I20260807 08:41:21.843871 4805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:34979 every 8 connection(s)
I20260807 08:41:21.844187 4703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:21.844867 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 4703
I20260807 08:41:21.844991 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:21.845253 11068 external_mini_cluster.cc:1428] Setting key 7a89344650db68f03ec19d799ee08bc5
I20260807 08:41:21.845667 4806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:21.849786 4806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d: Bootstrap starting.
I20260807 08:41:21.850605 4806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:21.850992 4806 log.cc:824] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:21.851951 4806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d: No bootstrap required, opened a new log
I20260807 08:41:21.852438 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:21.846057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.10.207.62:34979)
0807 08:41:21.846296 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:21.846304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:21.846334 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:41:21.847160 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:21.847172 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:21.847278 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:21.847425 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:21.847738 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.848725 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.849924 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:21.850403 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:21.850589 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:21.851479 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:21.851486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:21.851486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:21.851789 (+ 303us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:21.851839 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:21.851855 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:21.851919 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:21.852054 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:41:21.852261 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260807 08:41:21.854298 4806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34979 } }
I20260807 08:41:21.854463 4806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:21.854489 4806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949ad7c3487b4f5d88b71e02ccc4979d, State: Initialized, Role: FOLLOWER
I20260807 08:41:21.854661 4806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34979 } }
I20260807 08:41:21.854768 4806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:21.854816 4806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:21.854885 4806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:21.855751 4806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34979 } }
I20260807 08:41:21.855899 4806 leader_election.cc:302] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 949ad7c3487b4f5d88b71e02ccc4979d; no voters:
I20260807 08:41:21.856185 4806 leader_election.cc:288] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:21.856236 4814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:21.856458 4814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [term 1 LEADER]: Becoming Leader. State: Replica: 949ad7c3487b4f5d88b71e02ccc4979d, State: Running, Role: LEADER
I20260807 08:41:21.856599 4806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:21.856623 4814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34979 } }
I20260807 08:41:21.857098 4816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 949ad7c3487b4f5d88b71e02ccc4979d. Latest consensus state: current_term: 1 leader_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34979 } } }
I20260807 08:41:21.857102 4815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949ad7c3487b4f5d88b71e02ccc4979d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 34979 } } }
I20260807 08:41:21.857407 4815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:21.857364 4816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:21.857865 4830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:21.858431 4830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:21.860246 4830 catalog_manager.cc:1347] Generated new cluster ID: 10524bbbedf04868b758ca664f4343da
I20260807 08:41:21.860308 4830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:21.868762 4830 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:21.869395 4830 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:21.874357 4830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d: Generated new TSK 0
I20260807 08:41:21.874608 4830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:21.884001 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34979
--builtin_ntp_servers=127.10.207.20:38647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:41:21.994877 4848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:21.995150 4848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:21.995187 4848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:21.996387 4848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:21.996454 4848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:21.996502 4848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:41:21.997964 4848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:21.998262 4848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:21.998665 4848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:22.004678 4859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:22.004664 4858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:22.005008 4864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:22.005316 4848 server_base.cc:1034] running on GCE node
I20260807 08:41:22.005559 4848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:22.005856 4848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:22.007025 4848 hybrid_clock.cc:648] HybridClock initialized: now 1786092082006996 us; error 46 us; skew 500 ppm
I20260807 08:41:22.008857 4848 webserver.cc:460] Webserver started at http://127.10.207.1:41781/ using document root <none> and password file <none>
I20260807 08:41:22.009217 4848 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:22.009316 4848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:22.009502 4848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:22.010866 4848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "28718408949040948c7f0000565b699a"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "f2640564122589bc09a8c2ed5843bb70"
server_key_iv: "667cc42ae13582e0e3a4cd39c0bc06c2"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.011212 4848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "28718408949040948c7f0000565b699a"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "f2640564122589bc09a8c2ed5843bb70"
server_key_iv: "667cc42ae13582e0e3a4cd39c0bc06c2"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.012753 4848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:22.013614 4872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.013885 4848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:22.014020 4848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "28718408949040948c7f0000565b699a"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "f2640564122589bc09a8c2ed5843bb70"
server_key_iv: "667cc42ae13582e0e3a4cd39c0bc06c2"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.014110 4848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:22.039804 4848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:22.040136 4848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:22.040305 4848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:22.040566 4848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:22.040926 4848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:22.040988 4848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.041028 4848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:22.041060 4848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.048733 4848 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:41385
I20260807 08:41:22.048775 5048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:41385 every 8 connection(s)
I20260807 08:41:22.049238 4848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:41:22.052256 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 4848
I20260807 08:41:22.052354 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:41:22.052542 11068 external_mini_cluster.cc:1428] Setting key d84e2f4e380fa3962382e8c77269915a
I20260807 08:41:22.053766 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34979
--builtin_ntp_servers=127.10.207.20:38647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:41:22.055371 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.050223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:52343 (local address 127.10.207.62:34979)
0807 08:41:22.050319 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.050322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.050589 (+ 267us) server_negotiation.cc:407] Connection header received
0807 08:41:22.051161 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.051162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.051179 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.051249 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:22.051941 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.052706 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.053822 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.054099 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.054177 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.055069 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:22.055071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:22.055072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:22.055076 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:22.055091 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:22.055095 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:22.055151 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.055252 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.055291 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:41:22.055577 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.050170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34979 (local address 127.10.207.1:52343)
0807 08:41:22.050432 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.050467 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.051051 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.051322 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.051325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.051333 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:22.051836 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.051840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.052815 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.052816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.053731 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.053734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.054187 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.054189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.054316 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.054928 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:22.054928 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:22.054956 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:22.054978 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:22.055154 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:22.055159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:22.055162 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.055238 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.055339 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260807 08:41:22.056205 5049 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34979
I20260807 08:41:22.056320 5049 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:22.056540 5049 heartbeater.cc:511] Master 127.10.207.62:34979 requested a full tablet report, sending...
I20260807 08:41:22.057180 4747 ts_manager.cc:194] Registered new tserver with Master: 28718408949040948c7f0000565b699a (127.10.207.1:41385)
I20260807 08:41:22.057994 4747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.1:52343
W20260807 08:41:22.141124 5053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:22.141346 5053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:22.141386 5053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:22.142560 5053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:22.142619 5053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:22.142661 5053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:41:22.143999 5053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:22.144250 5053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:22.144534 5053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:22.149935 5060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:22.149935 5065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:22.150035 5062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:22.150167 5053 server_base.cc:1034] running on GCE node
I20260807 08:41:22.150508 5053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:22.150828 5053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:22.151966 5053 hybrid_clock.cc:648] HybridClock initialized: now 1786092082151952 us; error 37 us; skew 500 ppm
I20260807 08:41:22.153232 5053 webserver.cc:460] Webserver started at http://127.10.207.2:40499/ using document root <none> and password file <none>
I20260807 08:41:22.153488 5053 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:22.153539 5053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:22.153682 5053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:22.154767 5053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "41aa345632f648a9b990add5e6cea082"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "d3a0f42479913d72eac5e3a365419325"
server_key_iv: "a83981bd9d988c29f2786dd94935dfde"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.155113 5053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "41aa345632f648a9b990add5e6cea082"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "d3a0f42479913d72eac5e3a365419325"
server_key_iv: "a83981bd9d988c29f2786dd94935dfde"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.156486 5053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:22.157389 5073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.157665 5053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:22.157796 5053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "41aa345632f648a9b990add5e6cea082"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "d3a0f42479913d72eac5e3a365419325"
server_key_iv: "a83981bd9d988c29f2786dd94935dfde"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.157878 5053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:22.168263 5053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:22.168524 5053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:22.168641 5053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:22.168828 5053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:22.169131 5053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:22.169176 5053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.169198 5053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:22.169210 5053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.176139 5053 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:35421
I20260807 08:41:22.176224 5246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:35421 every 8 connection(s)
I20260807 08:41:22.176563 5053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:41:22.181217 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 5053
I20260807 08:41:22.181339 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:41:22.181542 11068 external_mini_cluster.cc:1428] Setting key f98ade0e53bb1758c0efc9894f6bb90f
I20260807 08:41:22.181945 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.177376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:42133 (local address 127.10.207.62:34979)
0807 08:41:22.177420 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.177422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.177667 (+ 245us) server_negotiation.cc:407] Connection header received
0807 08:41:22.178073 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.178074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.178091 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.178162 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:22.178884 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.179411 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.180365 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.180566 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.180642 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.181525 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:22.181527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:22.181527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:22.181533 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:22.181550 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:22.181554 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:22.181640 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.181811 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.181857 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:41:22.182112 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.177351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34979 (local address 127.10.207.2:42133)
0807 08:41:22.177538 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.177570 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.177980 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.178217 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.178220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.178227 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:22.178804 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.178809 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.179495 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.179495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.180277 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.180279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.180658 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.180659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.180761 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.181287 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:22.181288 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:22.181331 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:22.181359 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:22.181657 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:22.181665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:22.181668 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.181797 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.181897 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260807 08:41:22.182752 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34979
--builtin_ntp_servers=127.10.207.20:38647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:41:22.182758 5249 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34979
I20260807 08:41:22.182883 5249 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:22.183096 5249 heartbeater.cc:511] Master 127.10.207.62:34979 requested a full tablet report, sending...
I20260807 08:41:22.183569 4747 ts_manager.cc:194] Registered new tserver with Master: 41aa345632f648a9b990add5e6cea082 (127.10.207.2:35421)
I20260807 08:41:22.183979 4747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.2:42133
W20260807 08:41:22.274557 5253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:22.274839 5253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:22.274876 5253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:22.276511 5253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:22.276571 5253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:22.276614 5253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:41:22.278344 5253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:34979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:22.278620 5253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:22.278913 5253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:22.284269 5269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:22.284232 5267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:22.284222 5253 server_base.cc:1034] running on GCE node
W20260807 08:41:22.284243 5265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:22.284694 5253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:22.284979 5253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:22.286182 5253 hybrid_clock.cc:648] HybridClock initialized: now 1786092082286169 us; error 44 us; skew 500 ppm
I20260807 08:41:22.287755 5253 webserver.cc:460] Webserver started at http://127.10.207.3:41359/ using document root <none> and password file <none>
I20260807 08:41:22.288057 5253 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:22.288136 5253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:22.288307 5253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:22.289390 5253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "5232eafb6f4c420b8cdb1a8b4d622040"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "e8bf1ae6a087a1dec0ef68e15807402c"
server_key_iv: "f20cc9d0f545abd3528ee3f28dd6ba64"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.289754 5253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "5232eafb6f4c420b8cdb1a8b4d622040"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "e8bf1ae6a087a1dec0ef68e15807402c"
server_key_iv: "f20cc9d0f545abd3528ee3f28dd6ba64"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.291383 5253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:22.292354 5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.292635 5253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:22.292773 5253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "5232eafb6f4c420b8cdb1a8b4d622040"
format_stamp: "Formatted at 2026-08-07 08:41:22 on dist-test-slave-h6n0"
server_key: "e8bf1ae6a087a1dec0ef68e15807402c"
server_key_iv: "f20cc9d0f545abd3528ee3f28dd6ba64"
server_key_version: "encryptionkey@0"
I20260807 08:41:22.292865 5253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:22.325551 5253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:22.325858 5253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:22.326018 5253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:22.326267 5253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:22.326746 5253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:22.326807 5253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.326840 5253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:22.326861 5253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:22.340530 5253 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:45621
I20260807 08:41:22.340672 5446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:45621 every 8 connection(s)
I20260807 08:41:22.341001 5253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:41:22.348409 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.341937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:41723 (local address 127.10.207.62:34979)
0807 08:41:22.342025 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.342028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.342318 (+ 290us) server_negotiation.cc:407] Connection header received
0807 08:41:22.343143 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.343145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.343159 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.343192 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:22.344061 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.344651 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.346570 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.346999 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.347100 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.348024 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:22.348025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:22.348026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:22.348031 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:22.348062 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:22.348067 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:22.348138 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.348247 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.348276 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:41:22.348515 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.341904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:34979 (local address 127.10.207.3:41723)
0807 08:41:22.342159 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.342204 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.342977 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.343369 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.343373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.343383 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:22.343967 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.343971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.344785 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.344799 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.346426 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.346431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.347104 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.347105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.347251 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.347855 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:22.347856 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:22.347892 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:22.347920 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:22.348146 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:22.348152 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:22.348155 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.348232 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.348323 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":128,"threads_started":1}
I20260807 08:41:22.349318 5447 heartbeater.cc:347] Connected to a master server at 127.10.207.62:34979
I20260807 08:41:22.349424 5447 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:22.349615 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 5253
I20260807 08:41:22.349632 5447 heartbeater.cc:511] Master 127.10.207.62:34979 requested a full tablet report, sending...
I20260807 08:41:22.349721 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:41:22.349964 11068 external_mini_cluster.cc:1428] Setting key c29530cc8aad8bf4eac542cb722d6a06
I20260807 08:41:22.350121 4747 ts_manager.cc:194] Registered new tserver with Master: 5232eafb6f4c420b8cdb1a8b4d622040 (127.10.207.3:45621)
I20260807 08:41:22.350651 4747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.3:41723
I20260807 08:41:22.350937 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:41:22.356513 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.352761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.10.207.62:34979)
0807 08:41:22.352810 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.352813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.352930 (+ 117us) server_negotiation.cc:407] Connection header received
0807 08:41:22.353041 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.353042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.353054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.353115 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:22.353364 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.353920 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.355139 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.355329 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.355400 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.356221 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:22.356222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:22.356223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:22.356228 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:22.356243 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:22.356247 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:22.356328 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.356410 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.356434 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:41:22.358709 4746 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:41:22.361537 4746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:41:22.369949 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.364303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35421 (local address 127.0.0.1:51466)
0807 08:41:22.364455 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.364498 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.364580 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.365123 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.365126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.365129 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:22.365206 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.365209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.366727 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.366729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.368156 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.368161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.369468 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.369469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.369563 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.369579 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.369648 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.369722 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260807 08:41:22.370337 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.364658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.10.207.2:35421)
0807 08:41:22.364919 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.364924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.364937 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:41:22.364986 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.364988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.365007 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.365105 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:22.365291 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.366629 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.368313 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.369231 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.369327 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.369427 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.370133 (+ 706us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.370202 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":130,"threads_started":1}
I20260807 08:41:22.371217 5141 tablet_service.cc:1467] Processing CreateTablet for tablet 293dd1794188471ea54c96c921bcbe83 (DEFAULT_TABLE table=test-table [id=9d340856ace74b428858143538dc9c7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:22.371680 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.364520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41385 (local address 127.0.0.1:35194)
0807 08:41:22.366769 (+ 2249us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.366782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.366845 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.367132 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.367133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.367135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:22.367197 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.367199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.368649 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.368650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.369918 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.369921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.371432 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.371434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.371499 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.371504 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.371551 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.371606 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2229,"thread_start_us":61,"threads_started":1}
I20260807 08:41:22.371773 5141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293dd1794188471ea54c96c921bcbe83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:22.371971 5467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.364669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35194 (local address 127.10.207.1:41385)
0807 08:41:22.365247 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.365251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.366910 (+ 1659us) server_negotiation.cc:407] Connection header received
0807 08:41:22.366963 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.366966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.366982 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.367059 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:22.367279 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.368556 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.370128 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.371232 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.371306 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.371397 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.371838 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.371879 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":108,"threads_started":1}
I20260807 08:41:22.373070 5465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.364478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.10.207.3:45621)
0807 08:41:22.368743 (+ 4265us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.368748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.368759 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:41:22.368796 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.368798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.368814 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.368895 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:22.369101 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.370800 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.372055 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.372653 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.372718 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.372774 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.372968 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.373002 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4220,"thread_start_us":90,"threads_started":1}
I20260807 08:41:22.373095 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.364313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:45621 (local address 127.0.0.1:35586)
0807 08:41:22.364732 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.364744 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.364801 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.368969 (+ 4168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.368970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.368972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:22.369021 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.369022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.370888 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.370889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.371952 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.371955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.372761 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.372763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.372843 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.372848 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.372955 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.373017 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":62,"threads_started":1}
I20260807 08:41:22.375129 4935 tablet_service.cc:1467] Processing CreateTablet for tablet 293dd1794188471ea54c96c921bcbe83 (DEFAULT_TABLE table=test-table [id=9d340856ace74b428858143538dc9c7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:22.375269 5344 tablet_service.cc:1467] Processing CreateTablet for tablet 293dd1794188471ea54c96c921bcbe83 (DEFAULT_TABLE table=test-table [id=9d340856ace74b428858143538dc9c7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:22.375566 4935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293dd1794188471ea54c96c921bcbe83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:22.375703 5344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293dd1794188471ea54c96c921bcbe83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:22.377895 5469 tablet_bootstrap.cc:492] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a: Bootstrap starting.
I20260807 08:41:22.378659 5469 tablet_bootstrap.cc:654] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:22.378975 5470 tablet_bootstrap.cc:492] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Bootstrap starting.
I20260807 08:41:22.379035 5469 log.cc:824] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:22.379593 5470 tablet_bootstrap.cc:654] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:22.379900 5470 log.cc:824] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:22.379931 5469 tablet_bootstrap.cc:492] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a: No bootstrap required, opened a new log
I20260807 08:41:22.379997 5469 ts_tablet_manager.cc:1397] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:41:22.379915 5474 tablet_bootstrap.cc:492] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Bootstrap starting.
I20260807 08:41:22.380734 5474 tablet_bootstrap.cc:654] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:22.380872 5470 tablet_bootstrap.cc:492] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: No bootstrap required, opened a new log
I20260807 08:41:22.380952 5470 ts_tablet_manager.cc:1397] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:41:22.381131 5474 log.cc:824] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:22.381968 5474 tablet_bootstrap.cc:492] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: No bootstrap required, opened a new log
I20260807 08:41:22.382052 5474 ts_tablet_manager.cc:1397] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:41:22.382692 5469 raft_consensus.cc:348] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.382860 5469 raft_consensus.cc:374] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:22.382905 5469 raft_consensus.cc:729] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28718408949040948c7f0000565b699a, State: Initialized, Role: FOLLOWER
I20260807 08:41:22.383066 5469 consensus_queue.cc:260] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.383323 5469 ts_tablet_manager.cc:1428] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 08:41:22.383455 5049 heartbeater.cc:503] Master 127.10.207.62:34979 was elected leader, sending a full tablet report...
I20260807 08:41:22.383811 5470 raft_consensus.cc:348] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.383986 5470 raft_consensus.cc:374] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:22.384032 5470 raft_consensus.cc:729] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41aa345632f648a9b990add5e6cea082, State: Initialized, Role: FOLLOWER
I20260807 08:41:22.384157 5470 consensus_queue.cc:260] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.384436 5470 ts_tablet_manager.cc:1428] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 08:41:22.384624 5249 heartbeater.cc:503] Master 127.10.207.62:34979 was elected leader, sending a full tablet report...
I20260807 08:41:22.385183 5474 raft_consensus.cc:348] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.385358 5474 raft_consensus.cc:374] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:22.385402 5474 raft_consensus.cc:729] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5232eafb6f4c420b8cdb1a8b4d622040, State: Initialized, Role: FOLLOWER
I20260807 08:41:22.385524 5474 consensus_queue.cc:260] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.385850 5474 ts_tablet_manager.cc:1428] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 08:41:22.385896 5447 heartbeater.cc:503] Master 127.10.207.62:34979 was elected leader, sending a full tablet report...
W20260807 08:41:22.427168 5250 tablet.cc:2369] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:41:22.550168 5050 tablet.cc:2369] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:41:22.591681 5448 tablet.cc:2369] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:41:22.605967 5479 raft_consensus.cc:482] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:22.606092 5479 raft_consensus.cc:504] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.606560 5479 leader_election.cc:288] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41aa345632f648a9b990add5e6cea082 (127.10.207.2:35421), 5232eafb6f4c420b8cdb1a8b4d622040 (127.10.207.3:45621)
I20260807 08:41:22.609915 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.606903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:33995 (local address 127.10.207.2:35421)
0807 08:41:22.606981 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.606983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.607131 (+ 148us) server_negotiation.cc:407] Connection header received
0807 08:41:22.607207 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.607209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.607226 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.607299 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:22.607478 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.607979 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.609099 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.609525 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.609614 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.609624 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.609797 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.609837 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:41:22.609947 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.606813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35421 (local address 127.10.207.1:33995)
0807 08:41:22.607050 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.607061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.607143 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.607310 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.607311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.607313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:22.607401 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.607404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.608084 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.608085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.608994 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.608996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.609615 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.609616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.609696 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.609701 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.609781 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.609837 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":27,"thread_start_us":143,"threads_started":1}
I20260807 08:41:22.610010 5465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.606809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:34493 (local address 127.10.207.3:45621)
0807 08:41:22.606905 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.606907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.607008 (+ 101us) server_negotiation.cc:407] Connection header received
0807 08:41:22.607228 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.607230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.607251 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.607360 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:22.607616 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.608149 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.609220 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.609604 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.609696 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.609706 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.609856 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.609904 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:41:22.610085 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.606750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:45621 (local address 127.10.207.1:34493)
0807 08:41:22.606900 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.606915 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:22.607000 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:22.607393 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:22.607394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:22.607400 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:22.607522 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.607525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.608232 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.608233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.609129 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:22.609131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.609696 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.609698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:22.609773 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.609778 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:22.609837 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:41:22.609888 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260807 08:41:22.610520 5169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293dd1794188471ea54c96c921bcbe83" candidate_uuid: "28718408949040948c7f0000565b699a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aa345632f648a9b990add5e6cea082" is_pre_election: true
I20260807 08:41:22.610733 5169 raft_consensus.cc:2454] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28718408949040948c7f0000565b699a in term 0.
I20260807 08:41:22.610754 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293dd1794188471ea54c96c921bcbe83" candidate_uuid: "28718408949040948c7f0000565b699a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" is_pre_election: true
I20260807 08:41:22.610949 5362 raft_consensus.cc:2454] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28718408949040948c7f0000565b699a in term 0.
I20260807 08:41:22.611027 4874 leader_election.cc:302] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28718408949040948c7f0000565b699a, 41aa345632f648a9b990add5e6cea082; no voters:
I20260807 08:41:22.611179 5479 raft_consensus.cc:2781] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:41:22.611261 5479 raft_consensus.cc:482] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:41:22.611291 5479 raft_consensus.cc:3037] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:22.612128 5479 raft_consensus.cc:504] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.612314 5479 leader_election.cc:288] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [CANDIDATE]: Term 1 election: Requested vote from peers 41aa345632f648a9b990add5e6cea082 (127.10.207.2:35421), 5232eafb6f4c420b8cdb1a8b4d622040 (127.10.207.3:45621)
I20260807 08:41:22.612548 5169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293dd1794188471ea54c96c921bcbe83" candidate_uuid: "28718408949040948c7f0000565b699a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aa345632f648a9b990add5e6cea082"
I20260807 08:41:22.612550 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293dd1794188471ea54c96c921bcbe83" candidate_uuid: "28718408949040948c7f0000565b699a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5232eafb6f4c420b8cdb1a8b4d622040"
I20260807 08:41:22.612641 5362 raft_consensus.cc:3037] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:22.612641 5169 raft_consensus.cc:3037] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:22.613572 5169 raft_consensus.cc:2454] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28718408949040948c7f0000565b699a in term 1.
I20260807 08:41:22.613648 5362 raft_consensus.cc:2454] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28718408949040948c7f0000565b699a in term 1.
I20260807 08:41:22.613843 4874 leader_election.cc:302] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28718408949040948c7f0000565b699a, 41aa345632f648a9b990add5e6cea082; no voters:
I20260807 08:41:22.614027 5479 raft_consensus.cc:2781] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:22.614287 5479 raft_consensus.cc:686] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [term 1 LEADER]: Becoming Leader. State: Replica: 28718408949040948c7f0000565b699a, State: Running, Role: LEADER
I20260807 08:41:22.614425 5479 consensus_queue.cc:237] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } }
I20260807 08:41:22.615574 4747 catalog_manager.cc:5495] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a reported cstate change: term changed from 0 to 1, leader changed from <none> to 28718408949040948c7f0000565b699a (127.10.207.1). New cstate: current_term: 1 leader_uuid: "28718408949040948c7f0000565b699a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28718408949040948c7f0000565b699a" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 41385 } health_report { overall_health: HEALTHY } } }
I20260807 08:41:22.657648 5467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.654691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35200 (local address 127.10.207.1:41385)
0807 08:41:22.654753 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:22.654755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:22.654766 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:41:22.654829 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:22.654829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:22.654848 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:22.654911 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:41:22.655099 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.655626 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.656714 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:22.656980 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:22.657074 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:22.657266 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:41:22.657414 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:41:22.657498 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:22.657535 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:41:22.657560 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:41:22.659888 5362 raft_consensus.cc:1261] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 1 FOLLOWER]: Refusing update from remote peer 28718408949040948c7f0000565b699a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:22.659963 5169 raft_consensus.cc:1261] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 1 FOLLOWER]: Refusing update from remote peer 28718408949040948c7f0000565b699a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:22.660320 5486 consensus_queue.cc:1035] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5232eafb6f4c420b8cdb1a8b4d622040" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 45621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:22.660490 5479 consensus_queue.cc:1035] T 293dd1794188471ea54c96c921bcbe83 P 28718408949040948c7f0000565b699a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41aa345632f648a9b990add5e6cea082" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:22.663731 5495 mvcc.cc:205] Tried to move back new op lower bound from 7315833170571325440 to 7315833170389839872. Current Snapshot: MvccSnapshot[applied={T|T < 7315833170571325440}]
I20260807 08:41:22.666174 4746 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:22.666292 4746 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:22.666685 5493 mvcc.cc:205] Tried to move back new op lower bound from 7315833170571325440 to 7315833170389839872. Current Snapshot: MvccSnapshot[applied={T|T < 7315833170571325440 or (T in {7315833170571325440})}]
I20260807 08:41:22.667393 4746 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 949ad7c3487b4f5d88b71e02ccc4979d: Sending DeleteTablet for 3 replicas of tablet 293dd1794188471ea54c96c921bcbe83
I20260807 08:41:22.667814 5344 tablet_service.cc:1514] Processing DeleteTablet for tablet 293dd1794188471ea54c96c921bcbe83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:22 UTC) from {username='slave'} at 127.0.0.1:35586
I20260807 08:41:22.668196 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 4848
I20260807 08:41:22.668205 5343 tablet_service.cc:1514] Processing DeleteTablet for tablet 293dd1794188471ea54c96c921bcbe83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:22 UTC) from {username='slave'} at 127.0.0.1:35586
I20260807 08:41:22.668211 5141 tablet_service.cc:1514] Processing DeleteTablet for tablet 293dd1794188471ea54c96c921bcbe83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:22 UTC) from {username='slave'} at 127.0.0.1:51466
I20260807 08:41:22.679416 5508 tablet_replica.cc:330] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: stopping tablet replica
I20260807 08:41:22.679590 5508 raft_consensus.cc:2229] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:22.679728 5508 raft_consensus.cc:2258] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:22.680302 5513 tablet_replica.cc:330] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: stopping tablet replica
I20260807 08:41:22.680423 5513 raft_consensus.cc:2229] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:22.680518 5513 raft_consensus.cc:2258] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:22.681833 5513 ts_tablet_manager.cc:1905] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:22.683873 5513 ts_tablet_manager.cc:1918] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:22.683975 5513 log.cc:1196] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/293dd1794188471ea54c96c921bcbe83
I20260807 08:41:22.684266 5513 ts_tablet_manager.cc:1939] T 293dd1794188471ea54c96c921bcbe83 P 41aa345632f648a9b990add5e6cea082: Deleting consensus metadata
I20260807 08:41:22.684813 4725 catalog_manager.cc:4841] TS 41aa345632f648a9b990add5e6cea082 (127.10.207.2:35421): tablet 293dd1794188471ea54c96c921bcbe83 (table test-table [id=9d340856ace74b428858143538dc9c7d]) successfully deleted
I20260807 08:41:22.684968 5508 ts_tablet_manager.cc:1905] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:22.686347 5508 ts_tablet_manager.cc:1918] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:22.686434 5508 log.cc:1196] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/293dd1794188471ea54c96c921bcbe83
I20260807 08:41:22.686741 5508 ts_tablet_manager.cc:1939] T 293dd1794188471ea54c96c921bcbe83 P 5232eafb6f4c420b8cdb1a8b4d622040: Deleting consensus metadata
I20260807 08:41:22.687211 4730 catalog_manager.cc:4841] TS 5232eafb6f4c420b8cdb1a8b4d622040 (127.10.207.3:45621): tablet 293dd1794188471ea54c96c921bcbe83 (table test-table [id=9d340856ace74b428858143538dc9c7d]) successfully deleted
W20260807 08:41:22.687454 4730 catalog_manager.cc:4808] TS 5232eafb6f4c420b8cdb1a8b4d622040 (127.10.207.3:45621): delete failed for tablet 293dd1794188471ea54c96c921bcbe83 because the tablet was not found. No further retry: Not found: Tablet not found: 293dd1794188471ea54c96c921bcbe83
W20260807 08:41:22.696171 4725 connection.cc:535] client connection to 127.10.207.1:41385 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:41:22.696322 4725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:41:22.696471 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 5053
I20260807 08:41:22.700156 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:22.699923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:41385 (local address 127.0.0.1:35212)
0807 08:41:22.700012 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:22.700090 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:41385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 08:41:22.700340 4725 catalog_manager.cc:4584] TS 28718408949040948c7f0000565b699a (127.10.207.1:41385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 293dd1794188471ea54c96c921bcbe83: Network error: Client connection negotiation failed: client connection to 127.10.207.1:41385: connect: Connection refused (error 111)
I20260807 08:41:22.712431 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 5253
I20260807 08:41:22.719980 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 4703
2026-08-07T08:41:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1035 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:22 dist-test-slave-h6n0 krb5kdc[5519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:22 dist-test-slave-h6n0 krb5kdc[5519](info): set up 2 sockets
Aug 07 08:41:22 dist-test-slave-h6n0 krb5kdc[5519](info): commencing operation
krb5kdc: starting...
W20260807 08:41:23.985893 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:24.117116 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43627
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:43627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:24.209158 5542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:24.209458 5542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:24.209496 5542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:24.210662 5542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:24.210732 5542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:24.210757 5542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:24.210779 5542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:24.210808 5542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:24.212392 5542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:43627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:43627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:24.212709 5542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:24.212976 5542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:24.219228 5547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.219271 5550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.219354 5548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:24.219717 5542 server_base.cc:1034] running on GCE node
I20260807 08:41:24.220006 5542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:24.220363 5542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:24.221541 5542 hybrid_clock.cc:648] HybridClock initialized: now 1786092084221517 us; error 49 us; skew 500 ppm
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:24.224982 5542 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:24.225569 5542 webserver.cc:460] Webserver started at http://127.10.207.62:35805/ using document root <none> and password file <none>
I20260807 08:41:24.225841 5542 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:24.225912 5542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:24.226070 5542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:24.227492 5542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "4a13ab86ff8d4ac88be05c05b9f77154"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "3e71d539b92096d514bee257b24be978"
server_key_iv: "7f90e0f60eaee5baf031a0d37e9e1a79"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.228009 5542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "4a13ab86ff8d4ac88be05c05b9f77154"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "3e71d539b92096d514bee257b24be978"
server_key_iv: "7f90e0f60eaee5baf031a0d37e9e1a79"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.230079 5542 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:41:24.231211 5563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.231426 5542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:41:24.231535 5542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "4a13ab86ff8d4ac88be05c05b9f77154"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "3e71d539b92096d514bee257b24be978"
server_key_iv: "7f90e0f60eaee5baf031a0d37e9e1a79"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.231644 5542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:24.245191 5542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:24.247340 5542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:24.247517 5542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:24.252404 5542 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:43627
I20260807 08:41:24.252485 5647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:43627 every 8 connection(s)
I20260807 08:41:24.252897 5542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:24.254097 5650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:24.256961 5650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154: Bootstrap starting.
I20260807 08:41:24.257720 5650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:24.258071 5650 log.cc:824] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:24.259152 5650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154: No bootstrap required, opened a new log
I20260807 08:41:24.261032 5650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43627 } }
I20260807 08:41:24.261236 5650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:24.261265 5650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a13ab86ff8d4ac88be05c05b9f77154, State: Initialized, Role: FOLLOWER
I20260807 08:41:24.261447 5650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43627 } }
I20260807 08:41:24.261523 5650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:24.261546 5650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:24.261622 5650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:24.262271 5650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43627 } }
I20260807 08:41:24.262377 5650 leader_election.cc:302] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a13ab86ff8d4ac88be05c05b9f77154; no voters:
I20260807 08:41:24.262667 5650 leader_election.cc:288] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:24.262727 5653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:24.262976 5653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [term 1 LEADER]: Becoming Leader. State: Replica: 4a13ab86ff8d4ac88be05c05b9f77154, State: Running, Role: LEADER
I20260807 08:41:24.263077 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 5542
I20260807 08:41:24.263105 5653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43627 } }
I20260807 08:41:24.263228 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:24.263473 11068 external_mini_cluster.cc:1428] Setting key 145bff13930abcff3e94c87d9861c352
I20260807 08:41:24.263535 5658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a13ab86ff8d4ac88be05c05b9f77154. Latest consensus state: current_term: 1 leader_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43627 } } }
I20260807 08:41:24.263638 5658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:24.263656 5650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:24.264087 5655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a13ab86ff8d4ac88be05c05b9f77154" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 43627 } } }
I20260807 08:41:24.264192 5655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:24.264676 5661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:24.265321 5661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:24.267527 5661 catalog_manager.cc:1347] Generated new cluster ID: 9165f5e744ce46bfa002a9230d55cac0
I20260807 08:41:24.267593 5661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:24.276242 5661 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:24.277124 5661 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:24.279359 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.264217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.10.207.62:43627)
0807 08:41:24.264430 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.264438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.264658 (+ 220us) server_negotiation.cc:407] Connection header received
0807 08:41:24.264932 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.264942 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.265013 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.265166 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:24.266197 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.267587 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.268898 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.269558 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.269672 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.273995 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:24.274000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:24.274001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:24.274015 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:24.277007 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.277506 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.277509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.277510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.277619 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.277850 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.277850 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.277851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.278159 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:24.278834 (+ 675us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.278922 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.279103 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260807 08:41:24.282780 5661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154: Generated new TSK 0
I20260807 08:41:24.283003 5661 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:41:24.317338 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43627
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:24.432929 5690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:24.433180 5690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:24.433208 5690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:24.434440 5690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:24.434489 5690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:24.434518 5690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:41:24.436323 5690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:24.436605 5690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:24.436864 5690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:24.442548 5697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.442564 5702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.442565 5696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:24.443630 5690 server_base.cc:1034] running on GCE node
I20260807 08:41:24.443883 5690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:24.444185 5690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:24.445361 5690 hybrid_clock.cc:648] HybridClock initialized: now 1786092084445335 us; error 47 us; skew 500 ppm
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:24.450911 5690 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:41:24.451470 5690 webserver.cc:460] Webserver started at http://127.10.207.1:41739/ using document root <none> and password file <none>
I20260807 08:41:24.451767 5690 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:24.451823 5690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:24.451974 5690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:24.453207 5690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "32444e76efab435c8154df20b0c30fd7"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "f90339b2d2b45242a0fb8b864ed8704f"
server_key_iv: "7c60db122beb63dc2bd7546acc3cdb73"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.453626 5690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "32444e76efab435c8154df20b0c30fd7"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "f90339b2d2b45242a0fb8b864ed8704f"
server_key_iv: "7c60db122beb63dc2bd7546acc3cdb73"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.455277 5690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:24.456292 5712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.456553 5690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:24.456673 5690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "32444e76efab435c8154df20b0c30fd7"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "f90339b2d2b45242a0fb8b864ed8704f"
server_key_iv: "7c60db122beb63dc2bd7546acc3cdb73"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.456760 5690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:24.469039 5690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:24.471607 5690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:24.471783 5690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:24.472064 5690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:24.472419 5690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:24.472471 5690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.472508 5690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:24.472548 5690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.480054 5690 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:43505
I20260807 08:41:24.480139 5898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:43505 every 8 connection(s)
I20260807 08:41:24.480706 5690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:41:24.484900 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 5690
I20260807 08:41:24.485044 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:41:24.485301 11068 external_mini_cluster.cc:1428] Setting key d3291398f89e78688ad1a1ac64f25a65
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:24.492622 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.481732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:40081 (local address 127.10.207.62:43627)
0807 08:41:24.481888 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.481891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.482121 (+ 230us) server_negotiation.cc:407] Connection header received
0807 08:41:24.482927 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.482929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.482945 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.483001 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:24.484320 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.484873 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.486202 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.486609 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.486718 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.489395 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:24.489397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:24.489397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:24.489405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:24.490970 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.491388 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.491389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.491390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.491468 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.491882 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.491883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.491883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.492000 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:24.492060 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.492461 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.492548 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 08:41:24.492969 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.481693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43627 (local address 127.10.207.1:40081)
0807 08:41:24.481958 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.482010 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.482797 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.483090 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.483095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.483908 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:24.484160 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.484170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.484986 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.484988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.486101 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.486104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.486763 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.486764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.486874 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.487590 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:24.487601 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:24.489288 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:24.491056 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:24.491065 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:24.491067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:24.491310 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:24.491579 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:24.491581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:24.491582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:24.491791 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:24.492076 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:24.492083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:24.492275 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.492432 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.492668 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":128,"threads_started":1}
I20260807 08:41:24.493742 5899 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43627
I20260807 08:41:24.493865 5899 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:24.494084 5899 heartbeater.cc:511] Master 127.10.207.62:43627 requested a full tablet report, sending...
I20260807 08:41:24.494809 5589 ts_manager.cc:194] Registered new tserver with Master: 32444e76efab435c8154df20b0c30fd7 (127.10.207.1:43505)
I20260807 08:41:24.495476 5589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:40081
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:41:24.515690 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43627
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:24.608721 5909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:24.609014 5909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:24.609114 5909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:24.610695 5909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:24.610759 5909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:24.610800 5909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:41:24.612255 5909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:24.612443 5909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:24.612622 5909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:24.617622 5918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.617624 5922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.617626 5914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:24.617988 5909 server_base.cc:1034] running on GCE node
I20260807 08:41:24.618213 5909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:24.618449 5909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:24.619591 5909 hybrid_clock.cc:648] HybridClock initialized: now 1786092084619565 us; error 47 us; skew 500 ppm
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:24.622143 5909 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:41:24.622661 5909 webserver.cc:460] Webserver started at http://127.10.207.2:41121/ using document root <none> and password file <none>
I20260807 08:41:24.622910 5909 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:24.622977 5909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:24.623136 5909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:24.624121 5909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "782a2c5b63844d298e917b961e1a06d4"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "460e6131b57e0a458b8c78fda2f364a0"
server_key_iv: "91828adea08991cb50c44ecc5e98faa5"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.624471 5909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "782a2c5b63844d298e917b961e1a06d4"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "460e6131b57e0a458b8c78fda2f364a0"
server_key_iv: "91828adea08991cb50c44ecc5e98faa5"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.625842 5909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:24.626678 5933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.626902 5909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:24.627014 5909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "782a2c5b63844d298e917b961e1a06d4"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "460e6131b57e0a458b8c78fda2f364a0"
server_key_iv: "91828adea08991cb50c44ecc5e98faa5"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.627123 5909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:24.638823 5909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:24.641634 5909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:24.641800 5909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:24.642040 5909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:24.642354 5909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:24.642408 5909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.642457 5909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:24.642483 5909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.650127 5909 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:41513
I20260807 08:41:24.650183 6124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:41513 every 8 connection(s)
I20260807 08:41:24.650501 5909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:41:24.652694 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 5909
I20260807 08:41:24.652817 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:41:24.653044 11068 external_mini_cluster.cc:1428] Setting key 6c244b1b9f54206fa1a652d788d94e8a
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:24.663718 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.651405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:44313 (local address 127.10.207.62:43627)
0807 08:41:24.651500 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.651503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.651727 (+ 224us) server_negotiation.cc:407] Connection header received
0807 08:41:24.652397 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.652399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.652413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.652491 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:24.654771 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.655302 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.656840 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.657057 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.657148 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.660197 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:24.660199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:24.660199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:24.660208 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:24.661843 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.662422 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.662423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.662424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.662480 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.662844 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.662845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.662845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.662966 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:24.663033 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.663469 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.663597 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:41:24.664064 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.651295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43627 (local address 127.10.207.2:44313)
0807 08:41:24.651547 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.651606 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.652275 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.653360 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.653366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.654296 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:24.654650 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.654658 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.655412 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.655413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.656738 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.656741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.657161 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.657164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.657312 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.657974 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:24.657984 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:24.660070 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:24.661946 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:24.661956 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:24.661959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:24.662325 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:24.662600 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:24.662601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:24.662601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:24.662759 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:24.663051 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:24.663060 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:24.663272 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.663424 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.663729 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
I20260807 08:41:24.664713 6125 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43627
I20260807 08:41:24.664863 6125 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:24.665144 6125 heartbeater.cc:511] Master 127.10.207.62:43627 requested a full tablet report, sending...
I20260807 08:41:24.665632 5589 ts_manager.cc:194] Registered new tserver with Master: 782a2c5b63844d298e917b961e1a06d4 (127.10.207.2:41513)
I20260807 08:41:24.665990 5589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:44313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:41:24.682963 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43627
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:24.774523 6132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:24.774840 6132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:24.774879 6132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:24.776052 6132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:24.776124 6132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:24.776170 6132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:41:24.777714 6132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:43627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:24.778008 6132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:24.778261 6132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:24.783754 6139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.783798 6140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:24.783805 6142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:24.784173 6132 server_base.cc:1034] running on GCE node
I20260807 08:41:24.784421 6132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:24.784690 6132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:24.785868 6132 hybrid_clock.cc:648] HybridClock initialized: now 1786092084785843 us; error 38 us; skew 500 ppm
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:24.788700 6132 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:41:24.789217 6132 webserver.cc:460] Webserver started at http://127.10.207.3:46383/ using document root <none> and password file <none>
I20260807 08:41:24.789476 6132 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:24.789542 6132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:24.789708 6132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:24.790913 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "6b9d7585a134462695d365e9c8e55938"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "bc7dc61b649b29f308173c587a7e31fb"
server_key_iv: "9f6ea9433412734ee9f9bb69a797c3e5"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.791303 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "6b9d7585a134462695d365e9c8e55938"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "bc7dc61b649b29f308173c587a7e31fb"
server_key_iv: "9f6ea9433412734ee9f9bb69a797c3e5"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.792754 6132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 08:41:24.793581 6153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.793788 6132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:24.793900 6132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "6b9d7585a134462695d365e9c8e55938"
format_stamp: "Formatted at 2026-08-07 08:41:24 on dist-test-slave-h6n0"
server_key: "bc7dc61b649b29f308173c587a7e31fb"
server_key_iv: "9f6ea9433412734ee9f9bb69a797c3e5"
server_key_version: "encryptionkey@0"
I20260807 08:41:24.793975 6132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:24.808266 6132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:24.810840 6132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:24.811033 6132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:24.811329 6132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:24.811707 6132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:24.811761 6132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.811803 6132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:24.811834 6132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:24.819576 6132 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:44395
I20260807 08:41:24.819633 6339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:44395 every 8 connection(s)
I20260807 08:41:24.819994 6132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:41:24 dist-test-slave-h6n0 krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092084, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:24.829281 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 6132
I20260807 08:41:24.829414 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:41:24.829624 11068 external_mini_cluster.cc:1428] Setting key 9657ec314eb103d9223d167250541bd1
I20260807 08:41:24.830901 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.820825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:53803 (local address 127.10.207.62:43627)
0807 08:41:24.820927 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.820929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.821115 (+ 186us) server_negotiation.cc:407] Connection header received
0807 08:41:24.821789 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.821791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.821808 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.821886 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:24.823016 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.823552 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.824637 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.824827 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.824886 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.827438 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:24.827440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:24.827440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:24.827447 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:24.828903 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.829531 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.829532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.829532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.829603 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.829937 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.829938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.829938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.830112 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:24.830221 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.830645 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.830749 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:41:24.831053 6342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.820785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:43627 (local address 127.10.207.3:53803)
0807 08:41:24.820983 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.821022 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.821616 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.821951 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.821956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.822667 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:24.822924 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.822930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.823641 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.823642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.824549 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.824552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.824908 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.824908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.825021 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.825608 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:24.825618 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:24.827329 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:24.828997 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:24.829004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:24.829007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:24.829381 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:24.829695 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:24.829696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:24.829696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:24.829852 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:24.830203 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:24.830211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:24.830460 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.830611 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.830792 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260807 08:41:24.831693 6340 heartbeater.cc:347] Connected to a master server at 127.10.207.62:43627
I20260807 08:41:24.831815 6340 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:24.832056 6340 heartbeater.cc:511] Master 127.10.207.62:43627 requested a full tablet report, sending...
I20260807 08:41:24.832509 5589 ts_manager.cc:194] Registered new tserver with Master: 6b9d7585a134462695d365e9c8e55938 (127.10.207.3:44395)
I20260807 08:41:24.832909 5589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:53803
I20260807 08:41:24.841988 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:41:24.850063 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.843710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.10.207.62:43627)
0807 08:41:24.843797 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.843800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.843931 (+ 131us) server_negotiation.cc:407] Connection header received
0807 08:41:24.843960 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.843961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.843971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.844087 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:24.844675 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.845137 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.845841 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.846030 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.846104 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.847489 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:24.847490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:24.847491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:24.847498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:24.848855 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.849243 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.849244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.849244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.849302 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:24.849525 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:24.849526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:24.849526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:24.849653 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:24.849764 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.849899 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.849963 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:41:24.852120 5589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:41:24.854040 5589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:41:24.861836 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.857196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41513 (local address 127.0.0.1:45124)
0807 08:41:24.857429 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.857469 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.857541 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.858099 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.858099 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.858102 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:24.858162 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.858164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.859223 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.859224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.860599 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.860603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.861520 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.861521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.861605 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.861611 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.861643 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.861718 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":61,"threads_started":1}
I20260807 08:41:24.861843 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.857371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.10.207.3:44395)
0807 08:41:24.857596 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.857602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.857673 (+ 71us) server_negotiation.cc:407] Connection header received
0807 08:41:24.857775 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.857778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.857793 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.857876 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:24.858179 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.859396 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.860770 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.861370 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.861460 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.861529 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.861703 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.861752 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":118,"threads_started":1}
I20260807 08:41:24.862082 6352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.857236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:44395 (local address 127.0.0.1:46660)
0807 08:41:24.857590 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.857605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.857688 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.857933 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.857937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.857941 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:24.858046 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.858050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.859521 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.859522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.860667 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.860670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.861463 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.861464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.861568 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.861588 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.861682 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.861739 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":118,"threads_started":1}
I20260807 08:41:24.862231 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.857475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:43505 (local address 127.0.0.1:56940)
0807 08:41:24.858075 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.858085 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.858141 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.858383 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.858384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.858387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:24.858422 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.858423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.859690 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.859691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.860812 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.860815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.861878 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.861879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.861946 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.861952 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.862010 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.862074 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":4,"thread_start_us":72,"threads_started":1}
I20260807 08:41:24.862327 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.857755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56940 (local address 127.10.207.1:43505)
0807 08:41:24.858018 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.858024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.858197 (+ 173us) server_negotiation.cc:407] Connection header received
0807 08:41:24.858248 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.858250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.858269 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.858356 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:24.858511 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.859569 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.860902 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.861704 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.861775 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.861837 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.862162 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.862218 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":119,"threads_started":1}
I20260807 08:41:24.862773 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.857305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.10.207.2:41513)
0807 08:41:24.857683 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.857687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.857696 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:41:24.857736 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.857739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.857754 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.858243 (+ 489us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:24.858315 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.859119 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.860704 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.861410 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.861794 (+ 384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.861850 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.861879 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.862697 (+ 818us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":106,"threads_started":1}
I20260807 08:41:24.863080 6239 tablet_service.cc:1467] Processing CreateTablet for tablet f1445c940a5c44d595bc8ce6ab31e377 (DEFAULT_TABLE table=test-table [id=09cb35b8c15c4768a930f547c25133b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:24.863152 5800 tablet_service.cc:1467] Processing CreateTablet for tablet f1445c940a5c44d595bc8ce6ab31e377 (DEFAULT_TABLE table=test-table [id=09cb35b8c15c4768a930f547c25133b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:24.863508 6239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1445c940a5c44d595bc8ce6ab31e377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:24.863538 5800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1445c940a5c44d595bc8ce6ab31e377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:24.863468 6014 tablet_service.cc:1467] Processing CreateTablet for tablet f1445c940a5c44d595bc8ce6ab31e377 (DEFAULT_TABLE table=test-table [id=09cb35b8c15c4768a930f547c25133b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:24.863785 6014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1445c940a5c44d595bc8ce6ab31e377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:24.867254 6361 tablet_bootstrap.cc:492] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7: Bootstrap starting.
I20260807 08:41:24.867421 6362 tablet_bootstrap.cc:492] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Bootstrap starting.
I20260807 08:41:24.867791 6363 tablet_bootstrap.cc:492] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Bootstrap starting.
I20260807 08:41:24.868077 6361 tablet_bootstrap.cc:654] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:24.868199 6362 tablet_bootstrap.cc:654] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:24.868486 6363 tablet_bootstrap.cc:654] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:24.868705 6361 log.cc:824] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:24.868830 6362 log.cc:824] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:24.869107 6363 log.cc:824] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:24.869769 6362 tablet_bootstrap.cc:492] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: No bootstrap required, opened a new log
I20260807 08:41:24.869869 6362 ts_tablet_manager.cc:1397] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:41:24.869942 6363 tablet_bootstrap.cc:492] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: No bootstrap required, opened a new log
I20260807 08:41:24.870040 6363 ts_tablet_manager.cc:1397] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:41:24.870927 6361 tablet_bootstrap.cc:492] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7: No bootstrap required, opened a new log
I20260807 08:41:24.871012 6361 ts_tablet_manager.cc:1397] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:41:24.871580 6363 raft_consensus.cc:348] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.871739 6363 raft_consensus.cc:374] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:24.871784 6363 raft_consensus.cc:729] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782a2c5b63844d298e917b961e1a06d4, State: Initialized, Role: FOLLOWER
I20260807 08:41:24.871889 6363 consensus_queue.cc:260] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.872180 6125 heartbeater.cc:503] Master 127.10.207.62:43627 was elected leader, sending a full tablet report...
I20260807 08:41:24.872069 6362 raft_consensus.cc:348] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.872257 6362 raft_consensus.cc:374] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:24.872313 6362 raft_consensus.cc:729] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9d7585a134462695d365e9c8e55938, State: Initialized, Role: FOLLOWER
I20260807 08:41:24.872181 6363 ts_tablet_manager.cc:1428] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:41:24.872478 6362 consensus_queue.cc:260] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.872852 6340 heartbeater.cc:503] Master 127.10.207.62:43627 was elected leader, sending a full tablet report...
I20260807 08:41:24.872831 6362 ts_tablet_manager.cc:1428] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:41:24.873317 6361 raft_consensus.cc:348] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.873499 6361 raft_consensus.cc:374] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:24.873551 6361 raft_consensus.cc:729] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32444e76efab435c8154df20b0c30fd7, State: Initialized, Role: FOLLOWER
I20260807 08:41:24.873694 6361 consensus_queue.cc:260] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.873992 6361 ts_tablet_manager.cc:1428] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:41:24.874001 5899 heartbeater.cc:503] Master 127.10.207.62:43627 was elected leader, sending a full tablet report...
I20260807 08:41:24.877758 6371 raft_consensus.cc:482] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:24.877846 6371 raft_consensus.cc:504] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.878146 6371 leader_election.cc:288] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 782a2c5b63844d298e917b961e1a06d4 (127.10.207.2:41513), 32444e76efab435c8154df20b0c30fd7 (127.10.207.1:43505)
I20260807 08:41:24.881494 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.878437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:53349 (local address 127.10.207.1:43505)
0807 08:41:24.878479 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.878481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.878491 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:41:24.878575 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.878576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.878589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.878640 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:24.878849 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.879700 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.880649 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.881108 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.881170 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.881178 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.881376 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.881429 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:41:24.881488 6342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.878291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:43505 (local address 127.10.207.3:53349)
0807 08:41:24.878411 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.878428 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.878480 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.878695 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.878696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.878698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:24.878772 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.878774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.879791 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.879792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.880547 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.880549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.881193 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.881194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.881291 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.881295 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.881359 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.881396 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 08:41:24.881569 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.878415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:43137 (local address 127.10.207.2:41513)
0807 08:41:24.878465 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.878467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.878675 (+ 208us) server_negotiation.cc:407] Connection header received
0807 08:41:24.878743 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.878744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.878754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.878816 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:24.879035 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.879748 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.880781 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.881191 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.881271 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.881279 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.881463 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.881503 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:41:24.881825 5830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1445c940a5c44d595bc8ce6ab31e377" candidate_uuid: "6b9d7585a134462695d365e9c8e55938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32444e76efab435c8154df20b0c30fd7" is_pre_election: true
I20260807 08:41:24.881958 5830 raft_consensus.cc:2454] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b9d7585a134462695d365e9c8e55938 in term 0.
I20260807 08:41:24.882004 6374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.878352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:41513 (local address 127.10.207.3:43137)
0807 08:41:24.878608 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.878618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:24.878677 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:24.878832 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:24.878834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:24.878836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:24.878925 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.878927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.879844 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.879845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.880686 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:24.880688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.881283 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.881284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:24.881371 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.881376 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:24.881436 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:41:24.881480 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":8,"thread_start_us":100,"threads_started":1}
I20260807 08:41:24.882212 6156 leader_election.cc:302] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32444e76efab435c8154df20b0c30fd7, 6b9d7585a134462695d365e9c8e55938; no voters:
I20260807 08:41:24.882370 6371 raft_consensus.cc:2781] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:41:24.882432 6371 raft_consensus.cc:482] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:41:24.882452 6371 raft_consensus.cc:3037] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:24.882370 6050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1445c940a5c44d595bc8ce6ab31e377" candidate_uuid: "6b9d7585a134462695d365e9c8e55938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "782a2c5b63844d298e917b961e1a06d4" is_pre_election: true
I20260807 08:41:24.882576 6050 raft_consensus.cc:2454] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b9d7585a134462695d365e9c8e55938 in term 0.
I20260807 08:41:24.883131 6371 raft_consensus.cc:504] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.883270 6371 leader_election.cc:288] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [CANDIDATE]: Term 1 election: Requested vote from peers 782a2c5b63844d298e917b961e1a06d4 (127.10.207.2:41513), 32444e76efab435c8154df20b0c30fd7 (127.10.207.1:43505)
I20260807 08:41:24.883450 6050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1445c940a5c44d595bc8ce6ab31e377" candidate_uuid: "6b9d7585a134462695d365e9c8e55938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "782a2c5b63844d298e917b961e1a06d4"
I20260807 08:41:24.883479 5830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1445c940a5c44d595bc8ce6ab31e377" candidate_uuid: "6b9d7585a134462695d365e9c8e55938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32444e76efab435c8154df20b0c30fd7"
I20260807 08:41:24.883540 6050 raft_consensus.cc:3037] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:24.883586 5830 raft_consensus.cc:3037] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:24.884186 6050 raft_consensus.cc:2454] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b9d7585a134462695d365e9c8e55938 in term 1.
I20260807 08:41:24.884387 6155 leader_election.cc:302] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b9d7585a134462695d365e9c8e55938, 782a2c5b63844d298e917b961e1a06d4; no voters:
I20260807 08:41:24.884526 5830 raft_consensus.cc:2454] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b9d7585a134462695d365e9c8e55938 in term 1.
I20260807 08:41:24.884572 6371 raft_consensus.cc:2781] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:24.884742 6371 raft_consensus.cc:686] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 1 LEADER]: Becoming Leader. State: Replica: 6b9d7585a134462695d365e9c8e55938, State: Running, Role: LEADER
I20260807 08:41:24.884855 6371 consensus_queue.cc:237] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } }
I20260807 08:41:24.885618 5587 catalog_manager.cc:5495] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b9d7585a134462695d365e9c8e55938 (127.10.207.3). New cstate: current_term: 1 leader_uuid: "6b9d7585a134462695d365e9c8e55938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b9d7585a134462695d365e9c8e55938" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 44395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 } health_report { overall_health: UNKNOWN } } }
I20260807 08:41:24.897552 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.895276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.10.207.3:44395)
0807 08:41:24.895351 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:24.895353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:24.895365 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:41:24.895390 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:24.895390 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:24.895404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:24.895478 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:41:24.895588 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.896039 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.896789 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:24.896998 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:24.897068 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:24.897223 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:41:24.897313 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:41:24.897414 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:24.897465 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 08:41:24.897475 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:41:24.899688 6050 raft_consensus.cc:1261] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 1 FOLLOWER]: Refusing update from remote peer 6b9d7585a134462695d365e9c8e55938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:24.899688 5830 raft_consensus.cc:1261] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 1 FOLLOWER]: Refusing update from remote peer 6b9d7585a134462695d365e9c8e55938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:24.900216 6371 consensus_queue.cc:1035] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32444e76efab435c8154df20b0c30fd7" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 43505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:24.900430 6375 consensus_queue.cc:1035] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "782a2c5b63844d298e917b961e1a06d4" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 41513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:41:24.901376 6126 tablet.cc:2369] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:41:24.903369 6381 mvcc.cc:205] Tried to move back new op lower bound from 7315833179745722368 to 7315833179689172992. Current Snapshot: MvccSnapshot[applied={T|T < 7315833179745722368}]
I20260807 08:41:24.903680 6384 mvcc.cc:205] Tried to move back new op lower bound from 7315833179745722368 to 7315833179689172992. Current Snapshot: MvccSnapshot[applied={T|T < 7315833179745722368}]
I20260807 08:41:24.904174 6386 mvcc.cc:205] Tried to move back new op lower bound from 7315833179745722368 to 7315833179689172992. Current Snapshot: MvccSnapshot[applied={T|T < 7315833179745722368}]
I20260807 08:41:24.906299 5587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:24.906380 5587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:24.907359 5587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a13ab86ff8d4ac88be05c05b9f77154: Sending DeleteTablet for 3 replicas of tablet f1445c940a5c44d595bc8ce6ab31e377
I20260807 08:41:24.907718 6014 tablet_service.cc:1514] Processing DeleteTablet for tablet f1445c940a5c44d595bc8ce6ab31e377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:24 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:45124
I20260807 08:41:24.907709 6239 tablet_service.cc:1514] Processing DeleteTablet for tablet f1445c940a5c44d595bc8ce6ab31e377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:24 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46660
I20260807 08:41:24.907800 5800 tablet_service.cc:1514] Processing DeleteTablet for tablet f1445c940a5c44d595bc8ce6ab31e377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:24 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:56940
I20260807 08:41:24.908125 6407 tablet_replica.cc:330] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7: stopping tablet replica
I20260807 08:41:24.908237 6407 raft_consensus.cc:2229] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:24.908345 6407 raft_consensus.cc:2258] T f1445c940a5c44d595bc8ce6ab31e377 P 32444e76efab435c8154df20b0c30fd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:24.908428 6405 tablet_replica.cc:330] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: stopping tablet replica
I20260807 08:41:24.908545 6405 raft_consensus.cc:2229] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:24.908551 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 5690
I20260807 08:41:24.930169 6406 tablet_replica.cc:330] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: stopping tablet replica
I20260807 08:41:24.930356 6406 raft_consensus.cc:2229] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:41:24.930495 6406 raft_consensus.cc:2258] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:24.930719 6405 raft_consensus.cc:2258] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:24.933104 6405 ts_tablet_manager.cc:1905] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:24.935225 6406 ts_tablet_manager.cc:1905] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:24.937490 6405 ts_tablet_manager.cc:1918] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:24.937567 6405 log.cc:1196] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/f1445c940a5c44d595bc8ce6ab31e377
I20260807 08:41:24.937881 6405 ts_tablet_manager.cc:1939] T f1445c940a5c44d595bc8ce6ab31e377 P 782a2c5b63844d298e917b961e1a06d4: Deleting consensus metadata
I20260807 08:41:24.944202 5565 catalog_manager.cc:4841] TS 782a2c5b63844d298e917b961e1a06d4 (127.10.207.2:41513): tablet f1445c940a5c44d595bc8ce6ab31e377 (table test-table [id=09cb35b8c15c4768a930f547c25133b0]) successfully deleted
I20260807 08:41:24.944779 6406 ts_tablet_manager.cc:1918] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:24.944983 6406 log.cc:1196] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/f1445c940a5c44d595bc8ce6ab31e377
I20260807 08:41:24.945410 6406 ts_tablet_manager.cc:1939] T f1445c940a5c44d595bc8ce6ab31e377 P 6b9d7585a134462695d365e9c8e55938: Deleting consensus metadata
I20260807 08:41:24.945961 5568 catalog_manager.cc:4841] TS 6b9d7585a134462695d365e9c8e55938 (127.10.207.3:44395): tablet f1445c940a5c44d595bc8ce6ab31e377 (table test-table [id=09cb35b8c15c4768a930f547c25133b0]) successfully deleted
W20260807 08:41:24.952159 5568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:43505 (error 108)
I20260807 08:41:24.952421 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 5909
I20260807 08:41:24.972994 6352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:24.972782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:43505 (local address 127.0.0.1:56948)
0807 08:41:24.972861 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:24.972938 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:43505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 08:41:24.973187 5568 catalog_manager.cc:4584] TS 32444e76efab435c8154df20b0c30fd7 (127.10.207.1:43505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1445c940a5c44d595bc8ce6ab31e377: Network error: Client connection negotiation failed: client connection to 127.10.207.1:43505: connect: Connection refused (error 111)
I20260807 08:41:24.977679 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 6132
I20260807 08:41:24.993006 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 5542
2026-08-07T08:41:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2276 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T08:41:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:25Z Disabled control of system clock
I20260807 08:41:25.032578 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:36593
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:36593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:41:25.124807 6413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:25.125058 6413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:25.125078 6413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:25.126308 6413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:25.126353 6413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:25.126367 6413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:25.126381 6413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:25.126394 6413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:25.128328 6413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:36593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:36593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:25.128628 6413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:25.128968 6413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:25.134588 6425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.134603 6421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.134604 6422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:25.135017 6413 server_base.cc:1034] running on GCE node
I20260807 08:41:25.135353 6413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:25.135716 6413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:25.136880 6413 hybrid_clock.cc:648] HybridClock initialized: now 1786092085136839 us; error 63 us; skew 500 ppm
I20260807 08:41:25.138602 6413 webserver.cc:460] Webserver started at http://127.10.207.62:46173/ using document root <none> and password file <none>
I20260807 08:41:25.138908 6413 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:25.138967 6413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:25.139277 6413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:25.140588 6413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "e6bf53106e0e485fae9b5671fb293abc"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "ae22b05af9f2bbb1236a6602e169042b"
server_key_iv: "084e0be215b714f4409c2d5d42524eff"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.141000 6413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "e6bf53106e0e485fae9b5671fb293abc"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "ae22b05af9f2bbb1236a6602e169042b"
server_key_iv: "084e0be215b714f4409c2d5d42524eff"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.142592 6413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 08:41:25.143504 6434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.143752 6413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:25.143855 6413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "e6bf53106e0e485fae9b5671fb293abc"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "ae22b05af9f2bbb1236a6602e169042b"
server_key_iv: "084e0be215b714f4409c2d5d42524eff"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.143934 6413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:25.164682 6413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:25.165058 6413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:25.165236 6413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:25.170519 6413 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:36593
I20260807 08:41:25.170645 6517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:36593 every 8 connection(s)
I20260807 08:41:25.171025 6413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:25.172158 6518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:25.174556 6518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc: Bootstrap starting.
I20260807 08:41:25.175112 6518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:25.175367 6518 log.cc:824] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:25.176098 6518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc: No bootstrap required, opened a new log
I20260807 08:41:25.177735 6518 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bf53106e0e485fae9b5671fb293abc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36593 } }
I20260807 08:41:25.177877 6518 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:25.177902 6518 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bf53106e0e485fae9b5671fb293abc, State: Initialized, Role: FOLLOWER
I20260807 08:41:25.178013 6518 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bf53106e0e485fae9b5671fb293abc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36593 } }
I20260807 08:41:25.178115 6518 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:25.178154 6518 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:25.178200 6518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:25.178273 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 6413
I20260807 08:41:25.178390 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:25.178645 11068 external_mini_cluster.cc:1428] Setting key 84089a70d3d8919b09404c28cb432e01
I20260807 08:41:25.178943 6518 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bf53106e0e485fae9b5671fb293abc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36593 } }
I20260807 08:41:25.179073 6518 leader_election.cc:302] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6bf53106e0e485fae9b5671fb293abc; no voters:
I20260807 08:41:25.179261 6518 leader_election.cc:288] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:25.179409 6525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:25.179548 6525 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [term 1 LEADER]: Becoming Leader. State: Replica: e6bf53106e0e485fae9b5671fb293abc, State: Running, Role: LEADER
I20260807 08:41:25.179569 6518 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:25.179639 6525 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bf53106e0e485fae9b5671fb293abc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36593 } }
I20260807 08:41:25.179992 6525 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6bf53106e0e485fae9b5671fb293abc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bf53106e0e485fae9b5671fb293abc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36593 } } }
I20260807 08:41:25.180092 6525 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:25.180286 6529 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6bf53106e0e485fae9b5671fb293abc. Latest consensus state: current_term: 1 leader_uuid: "e6bf53106e0e485fae9b5671fb293abc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bf53106e0e485fae9b5671fb293abc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 36593 } } }
I20260807 08:41:25.180420 6529 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:25.181701 6548 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:25.181768 6548 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:25.181828 6539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:25.182013 6539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:25.183842 6539 catalog_manager.cc:1347] Generated new cluster ID: 33901dc765564098b11812c913d546ce
I20260807 08:41:25.183898 6539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:25.185503 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.179177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.10.207.62:36593)
0807 08:41:25.179439 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.179450 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.179512 (+ 62us) server_negotiation.cc:407] Connection header received
0807 08:41:25.179694 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.179703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.179813 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.179984 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:25.180439 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.181943 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.183148 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.183758 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.183892 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.184698 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:25.184702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:25.184703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:25.184935 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:25.184982 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:25.184997 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:25.185067 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.185169 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.185250 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260807 08:41:25.189417 6539 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:25.190169 6539 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:25.197700 6539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc: Generated new TSK 0
I20260807 08:41:25.197865 6539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:25.201777 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36593
--builtin_ntp_servers=127.10.207.20:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:41:25.295879 6551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:25.296149 6551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:25.296188 6551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:25.297394 6551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:25.297459 6551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:25.297506 6551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:41:25.299034 6551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:25.299319 6551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:25.299705 6551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:25.305598 6559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.305610 6560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.305912 6562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:25.306300 6551 server_base.cc:1034] running on GCE node
I20260807 08:41:25.306565 6551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:25.306857 6551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:25.308029 6551 hybrid_clock.cc:648] HybridClock initialized: now 1786092085308019 us; error 30 us; skew 500 ppm
I20260807 08:41:25.309576 6551 webserver.cc:460] Webserver started at http://127.10.207.1:42683/ using document root <none> and password file <none>
I20260807 08:41:25.309913 6551 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:25.309968 6551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:25.310114 6551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:25.311412 6551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "4cd0d8913670a37edaff89dc9d2c4409"
server_key_iv: "1d4648fc689f937371f4cb8750972a02"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.311817 6551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "4cd0d8913670a37edaff89dc9d2c4409"
server_key_iv: "1d4648fc689f937371f4cb8750972a02"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.313292 6551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:25.314191 6571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.314565 6551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:25.314687 6551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "4cd0d8913670a37edaff89dc9d2c4409"
server_key_iv: "1d4648fc689f937371f4cb8750972a02"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.314775 6551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:25.325269 6551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:25.325630 6551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:25.325788 6551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:25.326037 6551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:25.326409 6551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:25.326462 6551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.326509 6551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:25.326565 6551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.334201 6551 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:39159
I20260807 08:41:25.334251 6739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:39159 every 8 connection(s)
I20260807 08:41:25.334718 6551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:41:25.338688 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 6551
I20260807 08:41:25.338824 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:41:25.339074 11068 external_mini_cluster.cc:1428] Setting key 66faf2bb1c5a8954f0d5a3f6b7066e23
I20260807 08:41:25.340312 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36593
--builtin_ntp_servers=127.10.207.20:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:41:25.340696 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.335527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:51489 (local address 127.10.207.62:36593)
0807 08:41:25.335601 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.335603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.335838 (+ 235us) server_negotiation.cc:407] Connection header received
0807 08:41:25.336385 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.336386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.336401 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.336460 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:25.337152 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.337691 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.338831 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.339139 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.339216 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.340274 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:25.340276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:25.340276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:25.340283 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:25.340300 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:25.340305 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:25.340349 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.340527 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.340581 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:41:25.341192 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.335515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:36593 (local address 127.10.207.1:51489)
0807 08:41:25.335722 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.335758 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.336284 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.336517 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.336520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.336539 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:25.337066 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.337070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.337779 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.337780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.338717 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.338720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.339246 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.339247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.339431 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.340095 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:25.340096 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:25.340136 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:25.340166 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:25.340376 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:25.340384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:25.340387 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.340811 (+ 424us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.340923 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
I20260807 08:41:25.341849 6740 heartbeater.cc:347] Connected to a master server at 127.10.207.62:36593
I20260807 08:41:25.341962 6740 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:25.342193 6740 heartbeater.cc:511] Master 127.10.207.62:36593 requested a full tablet report, sending...
I20260807 08:41:25.342849 6462 ts_manager.cc:194] Registered new tserver with Master: c5835fc5c4fa47f6be4c71c94c8dbd15 (127.10.207.1:39159)
I20260807 08:41:25.343498 6462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.1:51489
W20260807 08:41:25.434221 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:25.434490 6743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:25.434525 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:25.435740 6743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:25.435810 6743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:25.435858 6743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:41:25.437284 6743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:25.437572 6743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:25.437901 6743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:25.443907 6755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.443928 6750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.443871 6752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:25.444236 6743 server_base.cc:1034] running on GCE node
I20260807 08:41:25.444458 6743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:25.444687 6743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:25.445897 6743 hybrid_clock.cc:648] HybridClock initialized: now 1786092085445879 us; error 27 us; skew 500 ppm
I20260807 08:41:25.447597 6743 webserver.cc:460] Webserver started at http://127.10.207.2:36605/ using document root <none> and password file <none>
I20260807 08:41:25.447878 6743 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:25.447947 6743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:25.448115 6743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:25.449501 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "c48aba2f4c24409582ac514a8853a8fa"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "6612371df8ceeb8eb0e12a8020264701"
server_key_iv: "fe9655244b7d347c620c42620605f694"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.449965 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "c48aba2f4c24409582ac514a8853a8fa"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "6612371df8ceeb8eb0e12a8020264701"
server_key_iv: "fe9655244b7d347c620c42620605f694"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.451591 6743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260807 08:41:25.452527 6767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.452839 6743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:25.452992 6743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "c48aba2f4c24409582ac514a8853a8fa"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "6612371df8ceeb8eb0e12a8020264701"
server_key_iv: "fe9655244b7d347c620c42620605f694"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.453087 6743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:25.472572 6743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:25.472885 6743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:25.473042 6743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:25.473279 6743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:25.473627 6743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:25.473685 6743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.473732 6743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:25.473775 6743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.481810 6743 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:35057
I20260807 08:41:25.481940 6943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:35057 every 8 connection(s)
I20260807 08:41:25.482255 6743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:41:25.487814 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 6743
I20260807 08:41:25.487937 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:41:25.488226 11068 external_mini_cluster.cc:1428] Setting key 4c381d37d2e4c1a49acb00aa0a0c6d2b
I20260807 08:41:25.488847 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.483312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:58275 (local address 127.10.207.62:36593)
0807 08:41:25.483426 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.483429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.483657 (+ 228us) server_negotiation.cc:407] Connection header received
0807 08:41:25.484198 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.484200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.484220 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.484287 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:25.485309 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.485843 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.486859 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.487051 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.487146 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.488409 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:25.488411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:25.488411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:25.488424 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:25.488444 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:25.488450 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:25.488514 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.488628 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.488753 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:41:25.489122 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.483248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:36593 (local address 127.10.207.2:58275)
0807 08:41:25.483514 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.483553 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.484086 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.484427 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.484431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.484439 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:25.485193 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.485199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.485945 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.485946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.486772 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.486774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.487161 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.487162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.487273 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.488199 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:25.488199 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:25.488232 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:25.488288 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:25.488525 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:25.488532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:25.488535 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.488616 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.488837 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":134,"threads_started":1}
I20260807 08:41:25.489367 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36593
--builtin_ntp_servers=127.10.207.20:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:41:25.489771 6944 heartbeater.cc:347] Connected to a master server at 127.10.207.62:36593
I20260807 08:41:25.489892 6944 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:25.490137 6944 heartbeater.cc:511] Master 127.10.207.62:36593 requested a full tablet report, sending...
I20260807 08:41:25.490653 6462 ts_manager.cc:194] Registered new tserver with Master: c48aba2f4c24409582ac514a8853a8fa (127.10.207.2:35057)
I20260807 08:41:25.491050 6462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.2:58275
W20260807 08:41:25.581936 6947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:25.582178 6947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:25.582201 6947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:25.583336 6947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:25.583380 6947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:25.583406 6947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:41:25.584743 6947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:36593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:25.584981 6947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:25.585345 6947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:25.591049 6955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.591043 6952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:25.591037 6953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:25.591379 6947 server_base.cc:1034] running on GCE node
I20260807 08:41:25.591645 6947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:25.591900 6947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:25.593065 6947 hybrid_clock.cc:648] HybridClock initialized: now 1786092085593052 us; error 39 us; skew 500 ppm
I20260807 08:41:25.594514 6947 webserver.cc:460] Webserver started at http://127.10.207.3:42127/ using document root <none> and password file <none>
I20260807 08:41:25.594856 6947 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:25.594928 6947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:25.595089 6947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:25.596329 6947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "79036186db274b1a9b26fde15039d314"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "cf64350cf481361016a7092458e0c556"
server_key_iv: "50d5eed0e365ebc9ad475e53107a9213"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.596727 6947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "79036186db274b1a9b26fde15039d314"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "cf64350cf481361016a7092458e0c556"
server_key_iv: "50d5eed0e365ebc9ad475e53107a9213"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.598187 6947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:25.599239 6967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.599625 6947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:41:25.599747 6947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "79036186db274b1a9b26fde15039d314"
format_stamp: "Formatted at 2026-08-07 08:41:25 on dist-test-slave-h6n0"
server_key: "cf64350cf481361016a7092458e0c556"
server_key_iv: "50d5eed0e365ebc9ad475e53107a9213"
server_key_version: "encryptionkey@0"
I20260807 08:41:25.599831 6947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:25.609022 6947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:25.609335 6947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:25.609494 6947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:25.609731 6947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:25.610091 6947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:25.610154 6947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.610193 6947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:25.610219 6947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:25.617722 6947 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:33911
I20260807 08:41:25.617770 7142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:33911 every 8 connection(s)
I20260807 08:41:25.618177 6947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:41:25.623986 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.619221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:55067 (local address 127.10.207.62:36593)
0807 08:41:25.619309 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.619311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.619600 (+ 289us) server_negotiation.cc:407] Connection header received
0807 08:41:25.620191 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.620193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.620207 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.620260 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:25.621006 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.621512 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.622496 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.622710 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.622781 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.623684 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:25.623686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:25.623686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:25.623691 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:25.623709 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:25.623713 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:25.623773 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.623865 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.623894 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:41:25.624221 7147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.619183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:36593 (local address 127.10.207.3:55067)
0807 08:41:25.619457 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.619501 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.620067 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.620336 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.620339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.620348 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:25.620889 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.620893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.621605 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.621606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.622407 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.622409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.622803 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.622804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.622924 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.623513 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:25.623514 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:25.623543 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:25.623566 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:25.623774 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:25.623782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:25.623785 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.623868 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.623971 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":116,"threads_started":1}
I20260807 08:41:25.624926 7143 heartbeater.cc:347] Connected to a master server at 127.10.207.62:36593
I20260807 08:41:25.625011 7143 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:25.625216 7143 heartbeater.cc:511] Master 127.10.207.62:36593 requested a full tablet report, sending...
I20260807 08:41:25.625669 6462 ts_manager.cc:194] Registered new tserver with Master: 79036186db274b1a9b26fde15039d314 (127.10.207.3:33911)
I20260807 08:41:25.626084 6462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.3:55067
I20260807 08:41:25.626351 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 6947
I20260807 08:41:25.626482 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:41:25.626739 11068 external_mini_cluster.cc:1428] Setting key e54e1f26deab1c3a3c8d230e72caef7c
I20260807 08:41:25.627580 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:41:25.632383 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.629207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.10.207.62:36593)
0807 08:41:25.629301 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.629303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.629411 (+ 108us) server_negotiation.cc:407] Connection header received
0807 08:41:25.629468 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.629469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.629477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.629524 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:25.629699 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.630190 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.631091 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.631265 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.631328 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.632106 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:25.632108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:25.632108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:25.632112 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:25.632124 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:25.632128 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:25.632198 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.632270 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.632300 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:41:25.634207 6462 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:41:25.635877 6462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:41:25.642489 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.638374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:33911 (local address 127.0.0.1:32996)
0807 08:41:25.638763 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.638799 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.638918 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.639197 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.639201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.639204 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:25.639269 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.639271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.640278 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.640278 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.641223 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.641225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.642230 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.642231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.642286 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.642294 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.642368 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.642420 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":106,"threads_started":1}
I20260807 08:41:25.642500 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.638476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.10.207.3:33911)
0807 08:41:25.638716 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.638724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.638874 (+ 150us) server_negotiation.cc:407] Connection header received
0807 08:41:25.639029 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.639032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.639047 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.639103 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:25.639356 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.640195 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.641325 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.642052 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.642095 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.642175 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.642384 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.642420 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":116,"threads_started":1}
I20260807 08:41:25.643229 7155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.638681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39159 (local address 127.0.0.1:59470)
0807 08:41:25.639165 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.639176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.639290 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.639638 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.639639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.639642 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:25.639694 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.639696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.640964 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.640965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.641899 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.641902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.642944 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.642946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.643019 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.643025 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.643097 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.643147 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":94,"threads_started":1}
I20260807 08:41:25.643227 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.638789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59470 (local address 127.10.207.1:39159)
0807 08:41:25.639050 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.639055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.639254 (+ 199us) server_negotiation.cc:407] Connection header received
0807 08:41:25.639410 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.639413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.639430 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.639520 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:25.639780 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.640870 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.642008 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.642838 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.642925 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.643024 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.643109 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.643150 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":115,"threads_started":1}
I20260807 08:41:25.643698 7158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.638374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:35057 (local address 127.0.0.1:52636)
0807 08:41:25.638846 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.638859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.638922 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.639397 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.639398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.639400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:25.639433 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.639435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.640866 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.640867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.642017 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.642020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.643330 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.643331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.643387 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.643391 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.643447 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.643486 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":49,"thread_start_us":118,"threads_started":1}
I20260807 08:41:25.643617 7037 tablet_service.cc:1467] Processing CreateTablet for tablet 1c69becd09634041a8103380f97fb298 (DEFAULT_TABLE table=test-table [id=e6a09c3550b242d689e0acd101f65cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:25.643989 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.638881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.10.207.2:35057)
0807 08:41:25.639155 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.639159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.639171 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:41:25.639209 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.639210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.639225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.639308 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:25.639530 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.640764 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.642114 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.642844 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.642925 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.643024 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.643834 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.643896 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":119,"threads_started":1}
I20260807 08:41:25.644106 6641 tablet_service.cc:1467] Processing CreateTablet for tablet 1c69becd09634041a8103380f97fb298 (DEFAULT_TABLE table=test-table [id=e6a09c3550b242d689e0acd101f65cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:25.644304 7037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c69becd09634041a8103380f97fb298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:25.644466 6641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c69becd09634041a8103380f97fb298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:25.647274 6846 tablet_service.cc:1467] Processing CreateTablet for tablet 1c69becd09634041a8103380f97fb298 (DEFAULT_TABLE table=test-table [id=e6a09c3550b242d689e0acd101f65cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:25.647691 6846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c69becd09634041a8103380f97fb298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:25.647861 7163 tablet_bootstrap.cc:492] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Bootstrap starting.
I20260807 08:41:25.647856 7165 tablet_bootstrap.cc:492] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15: Bootstrap starting.
I20260807 08:41:25.648736 7165 tablet_bootstrap.cc:654] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:25.648736 7163 tablet_bootstrap.cc:654] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:25.649159 7165 log.cc:824] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:25.649209 7163 log.cc:824] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:25.650184 7165 tablet_bootstrap.cc:492] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15: No bootstrap required, opened a new log
I20260807 08:41:25.650203 7163 tablet_bootstrap.cc:492] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: No bootstrap required, opened a new log
I20260807 08:41:25.650274 7165 ts_tablet_manager.cc:1397] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:41:25.650280 7163 ts_tablet_manager.cc:1397] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:41:25.650936 7168 tablet_bootstrap.cc:492] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Bootstrap starting.
I20260807 08:41:25.651700 7168 tablet_bootstrap.cc:654] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:25.652041 7168 log.cc:824] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:25.652906 7168 tablet_bootstrap.cc:492] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: No bootstrap required, opened a new log
I20260807 08:41:25.653000 7168 ts_tablet_manager.cc:1397] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:41:25.652917 7165 raft_consensus.cc:348] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.653126 7165 raft_consensus.cc:374] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:25.653172 7165 raft_consensus.cc:729] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5835fc5c4fa47f6be4c71c94c8dbd15, State: Initialized, Role: FOLLOWER
I20260807 08:41:25.653302 7165 consensus_queue.cc:260] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.653225 7163 raft_consensus.cc:348] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.653401 7163 raft_consensus.cc:374] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:25.653460 7163 raft_consensus.cc:729] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79036186db274b1a9b26fde15039d314, State: Initialized, Role: FOLLOWER
I20260807 08:41:25.653622 7165 ts_tablet_manager.cc:1428] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 08:41:25.653680 6740 heartbeater.cc:503] Master 127.10.207.62:36593 was elected leader, sending a full tablet report...
I20260807 08:41:25.653643 7163 consensus_queue.cc:260] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.654013 7143 heartbeater.cc:503] Master 127.10.207.62:36593 was elected leader, sending a full tablet report...
I20260807 08:41:25.653949 7163 ts_tablet_manager.cc:1428] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:41:25.655668 7168 raft_consensus.cc:348] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.655783 7168 raft_consensus.cc:374] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:25.655803 7168 raft_consensus.cc:729] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48aba2f4c24409582ac514a8853a8fa, State: Initialized, Role: FOLLOWER
I20260807 08:41:25.655962 7168 consensus_queue.cc:260] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.656229 7168 ts_tablet_manager.cc:1428] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 08:41:25.656332 6944 heartbeater.cc:503] Master 127.10.207.62:36593 was elected leader, sending a full tablet report...
I20260807 08:41:25.659406 7174 raft_consensus.cc:482] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:25.659492 7174 raft_consensus.cc:504] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.659816 7174 leader_election.cc:288] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79036186db274b1a9b26fde15039d314 (127.10.207.3:33911), c5835fc5c4fa47f6be4c71c94c8dbd15 (127.10.207.1:39159)
I20260807 08:41:25.663043 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.659979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:33911 (local address 127.10.207.2:60337)
0807 08:41:25.660042 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.660055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.660104 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.660287 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.660288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.660290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:25.660368 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.660370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.661217 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.661218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.662059 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.662062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.662772 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.662773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.662846 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.662850 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.662914 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.662959 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:41:25.663105 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.660090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:60337 (local address 127.10.207.3:33911)
0807 08:41:25.660144 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.660146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.660157 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:41:25.660187 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.660188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.660201 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.660269 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:25.660469 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.661126 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.662152 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.662694 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.662754 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.662762 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.662935 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.662966 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:41:25.663416 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.660338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:60285 (local address 127.10.207.1:39159)
0807 08:41:25.660401 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.660404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.660428 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:41:25.660453 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.660454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.660464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.660520 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:25.660793 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.661363 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.662446 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.662949 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.663032 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.663041 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.663298 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.663357 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:41:25.663416 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.660109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39159 (local address 127.10.207.2:60285)
0807 08:41:25.660310 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.660320 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:25.660365 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:25.660596 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:25.660597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:25.660600 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:25.660677 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.660679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.661454 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.661455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.662353 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:25.662355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.663047 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.663048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:25.663229 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.663235 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:41:25.663290 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:41:25.663336 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":122,"threads_started":1}
I20260807 08:41:25.663437 7070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c69becd09634041a8103380f97fb298" candidate_uuid: "c48aba2f4c24409582ac514a8853a8fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79036186db274b1a9b26fde15039d314" is_pre_election: true
I20260807 08:41:25.663606 7070 raft_consensus.cc:2454] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48aba2f4c24409582ac514a8853a8fa in term 0.
I20260807 08:41:25.663730 6675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c69becd09634041a8103380f97fb298" candidate_uuid: "c48aba2f4c24409582ac514a8853a8fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" is_pre_election: true
I20260807 08:41:25.663826 6769 leader_election.cc:302] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79036186db274b1a9b26fde15039d314, c48aba2f4c24409582ac514a8853a8fa; no voters:
I20260807 08:41:25.663882 6675 raft_consensus.cc:2454] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c48aba2f4c24409582ac514a8853a8fa in term 0.
I20260807 08:41:25.663973 7174 raft_consensus.cc:2781] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:41:25.664031 7174 raft_consensus.cc:482] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:41:25.664048 7174 raft_consensus.cc:3037] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:25.664708 7174 raft_consensus.cc:504] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.664839 7174 leader_election.cc:288] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [CANDIDATE]: Term 1 election: Requested vote from peers 79036186db274b1a9b26fde15039d314 (127.10.207.3:33911), c5835fc5c4fa47f6be4c71c94c8dbd15 (127.10.207.1:39159)
I20260807 08:41:25.665030 7070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c69becd09634041a8103380f97fb298" candidate_uuid: "c48aba2f4c24409582ac514a8853a8fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79036186db274b1a9b26fde15039d314"
I20260807 08:41:25.665064 6675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c69becd09634041a8103380f97fb298" candidate_uuid: "c48aba2f4c24409582ac514a8853a8fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15"
I20260807 08:41:25.665138 7070 raft_consensus.cc:3037] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:25.665146 6675 raft_consensus.cc:3037] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:25.665963 6675 raft_consensus.cc:2454] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48aba2f4c24409582ac514a8853a8fa in term 1.
I20260807 08:41:25.666046 7070 raft_consensus.cc:2454] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c48aba2f4c24409582ac514a8853a8fa in term 1.
I20260807 08:41:25.666148 6769 leader_election.cc:302] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c48aba2f4c24409582ac514a8853a8fa, c5835fc5c4fa47f6be4c71c94c8dbd15; no voters:
I20260807 08:41:25.666277 7174 raft_consensus.cc:2781] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:25.666427 7174 raft_consensus.cc:686] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 1 LEADER]: Becoming Leader. State: Replica: c48aba2f4c24409582ac514a8853a8fa, State: Running, Role: LEADER
I20260807 08:41:25.666556 7174 consensus_queue.cc:237] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } }
I20260807 08:41:25.667336 6458 catalog_manager.cc:5495] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa reported cstate change: term changed from 0 to 1, leader changed from <none> to c48aba2f4c24409582ac514a8853a8fa (127.10.207.2). New cstate: current_term: 1 leader_uuid: "c48aba2f4c24409582ac514a8853a8fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c48aba2f4c24409582ac514a8853a8fa" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 35057 } health_report { overall_health: HEALTHY } } }
I20260807 08:41:25.680069 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.677544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.10.207.2:35057)
0807 08:41:25.677606 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:25.677608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:25.677618 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:41:25.677647 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:25.677647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:25.677666 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:25.677740 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:41:25.677871 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.678296 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.679159 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:25.679368 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:25.679430 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:25.679690 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:41:25.679784 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:41:25.679957 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:25.679991 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 08:41:25.680003 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:41:25.682089 6675 raft_consensus.cc:1261] T 1c69becd09634041a8103380f97fb298 P c5835fc5c4fa47f6be4c71c94c8dbd15 [term 1 FOLLOWER]: Refusing update from remote peer c48aba2f4c24409582ac514a8853a8fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:25.682093 7070 raft_consensus.cc:1261] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 1 FOLLOWER]: Refusing update from remote peer c48aba2f4c24409582ac514a8853a8fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:25.682509 7174 consensus_queue.cc:1035] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "79036186db274b1a9b26fde15039d314" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 33911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:25.682703 7177 consensus_queue.cc:1035] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5835fc5c4fa47f6be4c71c94c8dbd15" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 39159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:25.686710 7184 mvcc.cc:205] Tried to move back new op lower bound from 7315833182951227392 to 7315833182890942464. Current Snapshot: MvccSnapshot[applied={T|T < 7315833182951227392}]
I20260807 08:41:25.688112 6458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:25.688198 6458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:25.689054 6458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6bf53106e0e485fae9b5671fb293abc: Sending DeleteTablet for 3 replicas of tablet 1c69becd09634041a8103380f97fb298
I20260807 08:41:25.689433 7037 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c69becd09634041a8103380f97fb298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:25 UTC) from {username='slave'} at 127.0.0.1:32996
I20260807 08:41:25.689489 6641 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c69becd09634041a8103380f97fb298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:25 UTC) from {username='slave'} at 127.0.0.1:59470
I20260807 08:41:25.689517 6846 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c69becd09634041a8103380f97fb298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:25 UTC) from {username='slave'} at 127.0.0.1:52636
I20260807 08:41:25.689769 7200 tablet_replica.cc:330] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: stopping tablet replica
I20260807 08:41:25.689877 7200 raft_consensus.cc:2229] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:25.689946 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 6551
I20260807 08:41:25.689993 7200 raft_consensus.cc:2258] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:25.694468 7200 ts_tablet_manager.cc:1905] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:25.696291 7200 ts_tablet_manager.cc:1918] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:25.696352 7200 log.cc:1196] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/1c69becd09634041a8103380f97fb298
I20260807 08:41:25.696563 7200 ts_tablet_manager.cc:1939] T 1c69becd09634041a8103380f97fb298 P 79036186db274b1a9b26fde15039d314: Deleting consensus metadata
I20260807 08:41:25.698015 6436 catalog_manager.cc:4841] TS 79036186db274b1a9b26fde15039d314 (127.10.207.3:33911): tablet 1c69becd09634041a8103380f97fb298 (table test-table [id=e6a09c3550b242d689e0acd101f65cb5]) successfully deleted
I20260807 08:41:25.702719 7201 tablet_replica.cc:330] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: stopping tablet replica
I20260807 08:41:25.702853 7201 raft_consensus.cc:2229] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:41:25.702984 7201 raft_consensus.cc:2258] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:25.705351 7201 ts_tablet_manager.cc:1905] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:25.707366 7201 ts_tablet_manager.cc:1918] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:25.707510 7201 log.cc:1196] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/1c69becd09634041a8103380f97fb298
I20260807 08:41:25.707856 7201 ts_tablet_manager.cc:1939] T 1c69becd09634041a8103380f97fb298 P c48aba2f4c24409582ac514a8853a8fa: Deleting consensus metadata
I20260807 08:41:25.708361 6435 catalog_manager.cc:4841] TS c48aba2f4c24409582ac514a8853a8fa (127.10.207.2:35057): tablet 1c69becd09634041a8103380f97fb298 (table test-table [id=e6a09c3550b242d689e0acd101f65cb5]) successfully deleted
W20260807 08:41:25.715366 6436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:39159 (error 108)
I20260807 08:41:25.715953 7158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:25.715787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:39159 (local address 127.0.0.1:59474)
0807 08:41:25.715861 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:25.715915 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:39159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:41:25.716051 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 6743
W20260807 08:41:25.719498 6436 catalog_manager.cc:4584] TS c5835fc5c4fa47f6be4c71c94c8dbd15 (127.10.207.1:39159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c69becd09634041a8103380f97fb298: Network error: Client connection negotiation failed: client connection to 127.10.207.1:39159: connect: Connection refused (error 111)
I20260807 08:41:25.725185 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 6947
I20260807 08:41:25.736224 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 6413
2026-08-07T08:41:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (740 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:25 dist-test-slave-h6n0 krb5kdc[7206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:25 dist-test-slave-h6n0 krb5kdc[7206](info): set up 2 sockets
Aug 07 08:41:25 dist-test-slave-h6n0 krb5kdc[7206](info): commencing operation
krb5kdc: starting...
W20260807 08:41:27.006435 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:27.147452 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:38705
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:38705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:27.241561 7227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:27.241876 7227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:27.241914 7227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:27.243606 7227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:27.243664 7227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:27.243678 7227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:27.243691 7227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:27.243705 7227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:27.245262 7227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:38705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:38705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:27.245512 7227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:27.245740 7227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:27.251151 7244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:27.251221 7238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:27.251157 7240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:27.251607 7227 server_base.cc:1034] running on GCE node
I20260807 08:41:27.251904 7227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:27.252256 7227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:27.253425 7227 hybrid_clock.cc:648] HybridClock initialized: now 1786092087253402 us; error 47 us; skew 500 ppm
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:27.256290 7227 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:27.256759 7227 webserver.cc:460] Webserver started at http://127.10.207.62:43225/ using document root <none> and password file <none>
I20260807 08:41:27.257020 7227 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:27.257094 7227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:27.257246 7227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:27.258378 7227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "734d29514f2f4b69a00b3805fe687990"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "f67dc35db7c28a6378c846f236e4600e"
server_key_iv: "476edb4e9228bcdb5e0b222e8ef069fd"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.258785 7227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "734d29514f2f4b69a00b3805fe687990"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "f67dc35db7c28a6378c846f236e4600e"
server_key_iv: "476edb4e9228bcdb5e0b222e8ef069fd"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.260180 7227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:27.260982 7258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.261205 7227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:41:27.261305 7227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "734d29514f2f4b69a00b3805fe687990"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "f67dc35db7c28a6378c846f236e4600e"
server_key_iv: "476edb4e9228bcdb5e0b222e8ef069fd"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.261385 7227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:27.282619 7227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:27.285009 7227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:27.285181 7227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:27.289428 7227 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:38705
I20260807 08:41:27.289489 7334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:38705 every 8 connection(s)
I20260807 08:41:27.289850 7227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:27.290966 7335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:27.293654 7335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990: Bootstrap starting.
I20260807 08:41:27.294061 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 7227
I20260807 08:41:27.294341 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:27.294442 7335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:27.294656 11068 external_mini_cluster.cc:1428] Setting key dc57e9779de8a04952e26cd81cce4a24
I20260807 08:41:27.294838 7335 log.cc:824] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:27.295820 7335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990: No bootstrap required, opened a new log
I20260807 08:41:27.297969 7335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734d29514f2f4b69a00b3805fe687990" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38705 } }
I20260807 08:41:27.298152 7335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:27.298206 7335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 734d29514f2f4b69a00b3805fe687990, State: Initialized, Role: FOLLOWER
I20260807 08:41:27.298345 7335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734d29514f2f4b69a00b3805fe687990" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38705 } }
I20260807 08:41:27.298430 7335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:27.298454 7335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:27.298521 7335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:27.299222 7335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734d29514f2f4b69a00b3805fe687990" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38705 } }
I20260807 08:41:27.299364 7335 leader_election.cc:302] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 734d29514f2f4b69a00b3805fe687990; no voters:
I20260807 08:41:27.299592 7335 leader_election.cc:288] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:27.299626 7342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:27.299749 7342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [term 1 LEADER]: Becoming Leader. State: Replica: 734d29514f2f4b69a00b3805fe687990, State: Running, Role: LEADER
I20260807 08:41:27.299842 7342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734d29514f2f4b69a00b3805fe687990" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38705 } }
I20260807 08:41:27.299897 7335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:27.300240 7343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 734d29514f2f4b69a00b3805fe687990. Latest consensus state: current_term: 1 leader_uuid: "734d29514f2f4b69a00b3805fe687990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734d29514f2f4b69a00b3805fe687990" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38705 } } }
I20260807 08:41:27.300343 7343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:27.300535 7344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "734d29514f2f4b69a00b3805fe687990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734d29514f2f4b69a00b3805fe687990" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 38705 } } }
I20260807 08:41:27.300608 7344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:27.300576 7349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:27.301267 7349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:27.303274 7349 catalog_manager.cc:1347] Generated new cluster ID: c3f622773d334e6d89bbf67edd883367
I20260807 08:41:27.303336 7349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:27.307752 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.295227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.10.207.62:38705)
0807 08:41:27.295447 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.295457 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.295496 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:41:27.296149 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.296155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.296207 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.296360 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:27.296997 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.298010 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.299075 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.299630 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.299762 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.303358 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:27.303362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:27.303363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:27.303376 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:27.305632 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.306162 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.306166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.306166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.306307 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.306607 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.306608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.306608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.306910 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:27.307067 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.307380 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.307531 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260807 08:41:27.308851 7349 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:27.309357 7349 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:27.325314 7349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990: Generated new TSK 0
I20260807 08:41:27.325527 7349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
Entry for principal kudu/127.10.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab.
I20260807 08:41:27.355650 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38705
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:27.446463 7372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:27.446753 7372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:27.446779 7372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:27.447926 7372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:27.447969 7372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:27.447995 7372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:41:27.449493 7372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.1.keytab
--principal=kudu/127.10.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:27.449751 7372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:27.450018 7372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:27.455699 7380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:27.455737 7383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:27.455695 7379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:27.456010 7372 server_base.cc:1034] running on GCE node
I20260807 08:41:27.456240 7372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:27.456518 7372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:27.457691 7372 hybrid_clock.cc:648] HybridClock initialized: now 1786092087457667 us; error 47 us; skew 500 ppm
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:27.460474 7372 init.cc:377] Logged in from keytab as kudu/127.10.207.1@KRBTEST.COM (short username kudu)
I20260807 08:41:27.460944 7372 webserver.cc:460] Webserver started at http://127.10.207.1:45873/ using document root <none> and password file <none>
I20260807 08:41:27.461202 7372 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:27.461268 7372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:27.461412 7372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:27.462627 7372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "0476cfb332b14d028bb0a82a56ca422b"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "46b3f76e75cb0a02bb4e9312332586f9"
server_key_iv: "98db3b86607ba1a2313ebacd07fb888c"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.462997 7372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "0476cfb332b14d028bb0a82a56ca422b"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "46b3f76e75cb0a02bb4e9312332586f9"
server_key_iv: "98db3b86607ba1a2313ebacd07fb888c"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.464452 7372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:41:27.465409 7394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.465623 7372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:27.465740 7372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "0476cfb332b14d028bb0a82a56ca422b"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "46b3f76e75cb0a02bb4e9312332586f9"
server_key_iv: "98db3b86607ba1a2313ebacd07fb888c"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.465828 7372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:27.482189 7372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:27.484323 7372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:27.484503 7372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:27.484753 7372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:27.485083 7372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:27.485137 7372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.485172 7372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:27.485215 7372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.492601 7372 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:45183
I20260807 08:41:27.492640 7560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:45183 every 8 connection(s)
I20260807 08:41:27.493165 7372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.1@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:27.502482 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 7372
I20260807 08:41:27.502661 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:41:27.502925 11068 external_mini_cluster.cc:1428] Setting key 6c99dd445fe120289164b938190facd3
I20260807 08:41:27.505189 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.494176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:52439 (local address 127.10.207.62:38705)
0807 08:41:27.494266 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.494271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.494491 (+ 220us) server_negotiation.cc:407] Connection header received
0807 08:41:27.495155 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.495156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.495169 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.495239 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:27.496348 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.496934 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.497981 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.498242 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.498334 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.501246 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:27.501248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:27.501249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:27.501259 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:27.503156 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.503813 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.503815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.503816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.503898 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.504256 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.504257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.504258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.504453 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:27.504554 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.504906 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.505039 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:41:27.505342 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.494103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38705 (local address 127.10.207.1:52439)
0807 08:41:27.494299 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.494348 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.495028 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.495387 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.495392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.495989 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:27.496211 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.496216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.497037 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.497038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.497873 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.497875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.498354 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.498355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.498470 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.499201 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:27.499209 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:27.501102 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:27.503263 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:27.503273 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:27.503276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:27.503674 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:27.503974 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:27.503975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:27.503975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:27.504147 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:27.504582 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:27.504590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:27.504745 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.504853 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.505022 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260807 08:41:27.505952 7561 heartbeater.cc:347] Connected to a master server at 127.10.207.62:38705
I20260807 08:41:27.506086 7561 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:27.506306 7561 heartbeater.cc:511] Master 127.10.207.62:38705 requested a full tablet report, sending...
I20260807 08:41:27.506948 7280 ts_manager.cc:194] Registered new tserver with Master: 0476cfb332b14d028bb0a82a56ca422b (127.10.207.1:45183)
I20260807 08:41:27.507624 7280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.1@KRBTEST.COM'} at 127.10.207.1:52439
WARNING: no policy specified for kudu/127.10.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
Entry for principal kudu/127.10.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab.
I20260807 08:41:27.534194 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38705
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:27.622323 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:27.622558 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:27.622586 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:27.623754 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:27.623804 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:27.623831 7572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:41:27.625967 7572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.2.keytab
--principal=kudu/127.10.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:27.626345 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:27.626667 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:27.632615 7583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:27.632752 7572 server_base.cc:1034] running on GCE node
W20260807 08:41:27.632583 7578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:27.632596 7580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:27.633113 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:27.633370 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:27.634580 7572 hybrid_clock.cc:648] HybridClock initialized: now 1786092087634566 us; error 38 us; skew 500 ppm
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:27.637209 7572 init.cc:377] Logged in from keytab as kudu/127.10.207.2@KRBTEST.COM (short username kudu)
I20260807 08:41:27.637708 7572 webserver.cc:460] Webserver started at http://127.10.207.2:42363/ using document root <none> and password file <none>
I20260807 08:41:27.638160 7572 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:27.638264 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:27.638440 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:27.639662 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "fab403ea94174496b01d41c0ba4f0dcc"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "eb3c96fdcce360e1013f03eafbcb81a3"
server_key_iv: "cc60a5054737e3112964a3dac7b10c3c"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.640054 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "fab403ea94174496b01d41c0ba4f0dcc"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "eb3c96fdcce360e1013f03eafbcb81a3"
server_key_iv: "cc60a5054737e3112964a3dac7b10c3c"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.641625 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:27.642520 7600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.642814 7572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:27.642928 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "fab403ea94174496b01d41c0ba4f0dcc"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "eb3c96fdcce360e1013f03eafbcb81a3"
server_key_iv: "cc60a5054737e3112964a3dac7b10c3c"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.643009 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:27.672091 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:27.674139 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:27.674314 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:27.674594 7572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:27.674950 7572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:27.675019 7572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.675052 7572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:27.675084 7572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.682232 7572 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:36367
I20260807 08:41:27.682304 7763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:36367 every 8 connection(s)
I20260807 08:41:27.682626 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.2@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:27.691439 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 7572
I20260807 08:41:27.691598 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:41:27.691905 11068 external_mini_cluster.cc:1428] Setting key c116bcd7e6c94acb2b1529c0d1e1ab89
I20260807 08:41:27.692759 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.683501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:55701 (local address 127.10.207.62:38705)
0807 08:41:27.683557 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.683560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.683830 (+ 270us) server_negotiation.cc:407] Connection header received
0807 08:41:27.684325 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.684327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.684339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.684402 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:27.685289 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.685841 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.686879 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.687075 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.687137 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.689433 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:27.689435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:27.689435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:27.689443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:27.690999 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.691583 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.691585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.691585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.691692 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.692057 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.692058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.692058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.692198 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:27.692285 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.692564 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.692669 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:41:27.692934 7766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.683480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38705 (local address 127.10.207.2:55701)
0807 08:41:27.683692 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.683729 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.684229 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.684467 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.684471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.685015 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:27.685202 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.685207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.685959 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.685960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.686787 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.686790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.687160 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.687160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.687257 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.687758 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:27.687766 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:27.689313 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:27.691083 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:27.691091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:27.691093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:27.691468 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:27.691809 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:27.691810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:27.691810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:27.691971 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:27.692283 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:27.692292 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:27.692423 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.692537 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.692705 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":129,"threads_started":1}
I20260807 08:41:27.693511 7764 heartbeater.cc:347] Connected to a master server at 127.10.207.62:38705
I20260807 08:41:27.693601 7764 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:27.693787 7764 heartbeater.cc:511] Master 127.10.207.62:38705 requested a full tablet report, sending...
I20260807 08:41:27.694267 7280 ts_manager.cc:194] Registered new tserver with Master: fab403ea94174496b01d41c0ba4f0dcc (127.10.207.2:36367)
I20260807 08:41:27.694842 7280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.2@KRBTEST.COM'} at 127.10.207.2:55701
WARNING: no policy specified for kudu/127.10.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
Entry for principal kudu/127.10.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab.
I20260807 08:41:27.721323 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38705
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:27.813691 7770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:27.813920 7770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:27.813944 7770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:27.815063 7770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:27.815110 7770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:27.815137 7770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:41:27.816558 7770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.3.keytab
--principal=kudu/127.10.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:38705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:27.816773 7770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:27.816969 7770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:27.822515 7783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:27.822515 7778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:27.822567 7776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:27.823041 7770 server_base.cc:1034] running on GCE node
I20260807 08:41:27.823304 7770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:27.823559 7770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:27.824707 7770 hybrid_clock.cc:648] HybridClock initialized: now 1786092087824695 us; error 47 us; skew 500 ppm
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:27.827601 7770 init.cc:377] Logged in from keytab as kudu/127.10.207.3@KRBTEST.COM (short username kudu)
I20260807 08:41:27.828142 7770 webserver.cc:460] Webserver started at http://127.10.207.3:40511/ using document root <none> and password file <none>
I20260807 08:41:27.828421 7770 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:27.828490 7770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:27.828640 7770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:27.829754 7770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "520d2aa5686a439a80481b5f3dc68662"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "3f3b32fc1e313e22f0867b5993a229ce"
server_key_iv: "fde7484893b07951de2e05bcf9270a83"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.830132 7770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "520d2aa5686a439a80481b5f3dc68662"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "3f3b32fc1e313e22f0867b5993a229ce"
server_key_iv: "fde7484893b07951de2e05bcf9270a83"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.831694 7770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:27.832551 7794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.832774 7770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:27.832890 7770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "520d2aa5686a439a80481b5f3dc68662"
format_stamp: "Formatted at 2026-08-07 08:41:27 on dist-test-slave-h6n0"
server_key: "3f3b32fc1e313e22f0867b5993a229ce"
server_key_iv: "fde7484893b07951de2e05bcf9270a83"
server_key_version: "encryptionkey@0"
I20260807 08:41:27.832967 7770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:27.849241 7770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:27.851600 7770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:27.851769 7770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:27.852088 7770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:27.852442 7770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:27.852495 7770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.852540 7770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:27.852568 7770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:27.860051 7770 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:37127
I20260807 08:41:27.860126 7957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:37127 every 8 connection(s)
I20260807 08:41:27.860491 7770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:41:27 dist-test-slave-h6n0 krb5kdc[7206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092087, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.3@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:27.868257 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 7770
I20260807 08:41:27.868378 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:41:27.868561 11068 external_mini_cluster.cc:1428] Setting key 151118d6341b1408daac5173b98803e4
I20260807 08:41:27.870956 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.861308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:54849 (local address 127.10.207.62:38705)
0807 08:41:27.861354 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.861357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.861587 (+ 230us) server_negotiation.cc:407] Connection header received
0807 08:41:27.862117 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.862119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.862132 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.862196 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:27.863276 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.863824 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.864934 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.865160 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.865222 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.867491 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:27.867493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:27.867493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:27.867500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:27.869371 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.869890 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.869891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.869892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.869954 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.870222 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.870223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.870223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.870353 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:27.870429 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.870764 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.870850 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:41:27.871205 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.861287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:38705 (local address 127.10.207.3:54849)
0807 08:41:27.861459 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.861499 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.862021 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.862306 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.862309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.862936 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:27.863170 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.863175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.863909 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.863910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.864860 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.864862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.865292 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.865294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.865448 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.865953 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:27.865962 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:27.867395 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:27.869458 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:27.869467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:27.869469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:27.869816 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:27.870026 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:27.870027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:27.870027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:27.870155 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:27.870419 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:27.870426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:27.870631 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.870742 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.870933 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260807 08:41:27.871932 7958 heartbeater.cc:347] Connected to a master server at 127.10.207.62:38705
I20260807 08:41:27.872044 7958 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:27.872241 7958 heartbeater.cc:511] Master 127.10.207.62:38705 requested a full tablet report, sending...
I20260807 08:41:27.872658 7280 ts_manager.cc:194] Registered new tserver with Master: 520d2aa5686a439a80481b5f3dc68662 (127.10.207.3:37127)
I20260807 08:41:27.873059 7280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.207.3@KRBTEST.COM'} at 127.10.207.3:54849
I20260807 08:41:27.880795 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:41:27.889006 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.882201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.10.207.62:38705)
0807 08:41:27.882277 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.882279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.882327 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:41:27.882375 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.882375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.882385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.882440 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:27.883029 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.883541 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.884341 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.884565 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.884668 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.886030 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:27.886031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:27.886032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:27.886039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:27.887776 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.888157 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.888158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.888159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.888217 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:27.888403 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:27.888403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:27.888404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:27.888572 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:27.888686 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.888851 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.888927 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:41:27.890913 7280 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:41:27.892926 7280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:41:27.900694 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.895927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:37127 (local address 127.0.0.1:41540)
0807 08:41:27.896183 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.896222 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.896302 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.896632 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.896636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.896639 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:27.896721 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.896723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.898333 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.898334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.899435 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.899438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.900350 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.900351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.900438 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.900454 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.900519 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.900602 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":67,"threads_started":1}
I20260807 08:41:27.900782 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.896383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.10.207.1:45183)
0807 08:41:27.896661 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.896668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.896964 (+ 296us) server_negotiation.cc:407] Connection header received
0807 08:41:27.897024 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.897026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.897039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.897124 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:27.897236 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.897918 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.899474 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.900283 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.900385 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.900466 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.900615 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.900671 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":133,"threads_started":1}
I20260807 08:41:27.901106 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.896110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46944 (local address 127.10.207.2:36367)
0807 08:41:27.896696 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.896700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.896711 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:41:27.896780 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.896783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.896804 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.897131 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:27.897212 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.898377 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.899755 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.900656 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.900766 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.900829 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.900960 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.901023 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":109,"threads_started":1}
I20260807 08:41:27.901154 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.895939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:36367 (local address 127.0.0.1:46944)
0807 08:41:27.896338 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.896352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.896414 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.896980 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.896981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.896984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:27.897043 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.897044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.898489 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.898491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.899636 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.899639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.900772 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.900773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.900847 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.900852 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.900998 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.901058 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":120,"threads_started":1}
I20260807 08:41:27.901418 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.896249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:45183 (local address 127.0.0.1:46190)
0807 08:41:27.896839 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.896849 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.896900 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.897123 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.897124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.897126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:27.897178 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.897180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.898039 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.898040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.899358 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.899361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.900385 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.900386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.900478 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.900484 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.900584 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.900686 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":7,"thread_start_us":89,"threads_started":1}
I20260807 08:41:27.901850 7975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.896013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.10.207.3:37127)
0807 08:41:27.896176 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.896181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.896395 (+ 214us) server_negotiation.cc:407] Connection header received
0807 08:41:27.896449 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.896451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.896469 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.896541 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:27.896824 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.898210 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.899514 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.900245 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.901645 (+ 1400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.901708 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.901733 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.901776 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260807 08:41:27.901973 7665 tablet_service.cc:1467] Processing CreateTablet for tablet 37096c86161645489aef532b3b116f3a (DEFAULT_TABLE table=test-table [id=36542faabf8d4363895661a844bf01ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:27.902390 7665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37096c86161645489aef532b3b116f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:27.902351 7466 tablet_service.cc:1467] Processing CreateTablet for tablet 37096c86161645489aef532b3b116f3a (DEFAULT_TABLE table=test-table [id=36542faabf8d4363895661a844bf01ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:27.902866 7466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37096c86161645489aef532b3b116f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:27.905356 7983 tablet_bootstrap.cc:492] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Bootstrap starting.
I20260807 08:41:27.906286 7983 tablet_bootstrap.cc:654] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:27.906795 7984 tablet_bootstrap.cc:492] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b: Bootstrap starting.
I20260807 08:41:27.907006 7857 tablet_service.cc:1467] Processing CreateTablet for tablet 37096c86161645489aef532b3b116f3a (DEFAULT_TABLE table=test-table [id=36542faabf8d4363895661a844bf01ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:27.907104 7983 log.cc:824] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:27.907375 7857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37096c86161645489aef532b3b116f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:27.907627 7984 tablet_bootstrap.cc:654] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:27.908183 7983 tablet_bootstrap.cc:492] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: No bootstrap required, opened a new log
I20260807 08:41:27.908254 7984 log.cc:824] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:27.908288 7983 ts_tablet_manager.cc:1397] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:41:27.909325 7984 tablet_bootstrap.cc:492] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b: No bootstrap required, opened a new log
I20260807 08:41:27.909415 7984 ts_tablet_manager.cc:1397] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:41:27.910297 7983 raft_consensus.cc:348] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.910497 7983 raft_consensus.cc:374] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:27.910601 7983 raft_consensus.cc:729] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab403ea94174496b01d41c0ba4f0dcc, State: Initialized, Role: FOLLOWER
I20260807 08:41:27.910780 7983 consensus_queue.cc:260] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.911140 7983 ts_tablet_manager.cc:1428] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:41:27.911159 7764 heartbeater.cc:503] Master 127.10.207.62:38705 was elected leader, sending a full tablet report...
I20260807 08:41:27.911859 7989 tablet_bootstrap.cc:492] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Bootstrap starting.
I20260807 08:41:27.911933 7984 raft_consensus.cc:348] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.912123 7984 raft_consensus.cc:374] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:27.912150 7984 raft_consensus.cc:729] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0476cfb332b14d028bb0a82a56ca422b, State: Initialized, Role: FOLLOWER
I20260807 08:41:27.912264 7984 consensus_queue.cc:260] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.912530 7984 ts_tablet_manager.cc:1428] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 08:41:27.912672 7561 heartbeater.cc:503] Master 127.10.207.62:38705 was elected leader, sending a full tablet report...
I20260807 08:41:27.912729 7989 tablet_bootstrap.cc:654] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:27.913407 7989 log.cc:824] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:27.914253 7989 tablet_bootstrap.cc:492] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: No bootstrap required, opened a new log
I20260807 08:41:27.914322 7989 ts_tablet_manager.cc:1397] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:41:27.916206 7989 raft_consensus.cc:348] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.916365 7989 raft_consensus.cc:374] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:27.916394 7989 raft_consensus.cc:729] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520d2aa5686a439a80481b5f3dc68662, State: Initialized, Role: FOLLOWER
I20260807 08:41:27.916530 7989 consensus_queue.cc:260] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.916801 7989 ts_tablet_manager.cc:1428] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:41:27.916851 7958 heartbeater.cc:503] Master 127.10.207.62:38705 was elected leader, sending a full tablet report...
W20260807 08:41:27.933214 7765 tablet.cc:2369] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:41:27.945717 7995 raft_consensus.cc:482] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:27.945869 7995 raft_consensus.cc:504] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.946285 7995 leader_election.cc:288] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fab403ea94174496b01d41c0ba4f0dcc (127.10.207.2:36367), 0476cfb332b14d028bb0a82a56ca422b (127.10.207.1:45183)
I20260807 08:41:27.949270 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.946608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:49115 (local address 127.10.207.2:36367)
0807 08:41:27.946676 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.946679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.946692 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:41:27.946738 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.946738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.946753 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.946818 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:27.947021 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.947494 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.948538 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.948919 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.949010 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.949019 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.949166 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.949206 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:41:27.949271 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.946470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:36367 (local address 127.10.207.3:49115)
0807 08:41:27.946594 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.946611 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.946681 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.946883 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.946885 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.946886 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:27.946944 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.946946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.947584 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.947585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.948450 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.948453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.949009 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.949010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.949094 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.949097 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.949160 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.949207 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 08:41:27.949697 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.946758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:50645 (local address 127.10.207.1:45183)
0807 08:41:27.946812 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.946814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.946941 (+ 127us) server_negotiation.cc:407] Connection header received
0807 08:41:27.947023 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.947028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.947041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.947139 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:27.947324 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.947853 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.948864 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.949278 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.949363 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.949374 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.949567 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.949615 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:41:27.949678 7697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37096c86161645489aef532b3b116f3a" candidate_uuid: "520d2aa5686a439a80481b5f3dc68662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab403ea94174496b01d41c0ba4f0dcc" is_pre_election: true
I20260807 08:41:27.949896 7697 raft_consensus.cc:2454] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520d2aa5686a439a80481b5f3dc68662 in term 0.
I20260807 08:41:27.949957 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.946624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:45183 (local address 127.10.207.3:50645)
0807 08:41:27.946830 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:27.946840 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:27.946951 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:27.947144 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:27.947145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:27.947147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:27.947246 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.947248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.947936 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.947937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.948769 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:27.948771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.949380 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.949381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:27.949464 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.949468 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:41:27.949808 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 08:41:27.949861 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
I20260807 08:41:27.950232 7795 leader_election.cc:302] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520d2aa5686a439a80481b5f3dc68662, fab403ea94174496b01d41c0ba4f0dcc; no voters:
I20260807 08:41:27.950333 7492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37096c86161645489aef532b3b116f3a" candidate_uuid: "520d2aa5686a439a80481b5f3dc68662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0476cfb332b14d028bb0a82a56ca422b" is_pre_election: true
I20260807 08:41:27.950431 7995 raft_consensus.cc:2781] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:41:27.950554 7995 raft_consensus.cc:482] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:41:27.950495 7492 raft_consensus.cc:2454] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520d2aa5686a439a80481b5f3dc68662 in term 0.
I20260807 08:41:27.950613 7995 raft_consensus.cc:3037] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:27.951327 7995 raft_consensus.cc:504] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.951483 7995 leader_election.cc:288] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [CANDIDATE]: Term 1 election: Requested vote from peers fab403ea94174496b01d41c0ba4f0dcc (127.10.207.2:36367), 0476cfb332b14d028bb0a82a56ca422b (127.10.207.1:45183)
I20260807 08:41:27.951740 7697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37096c86161645489aef532b3b116f3a" candidate_uuid: "520d2aa5686a439a80481b5f3dc68662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab403ea94174496b01d41c0ba4f0dcc"
I20260807 08:41:27.951766 7492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37096c86161645489aef532b3b116f3a" candidate_uuid: "520d2aa5686a439a80481b5f3dc68662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0476cfb332b14d028bb0a82a56ca422b"
I20260807 08:41:27.951845 7492 raft_consensus.cc:3037] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:27.951845 7697 raft_consensus.cc:3037] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:27.952934 7492 raft_consensus.cc:2454] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520d2aa5686a439a80481b5f3dc68662 in term 1.
I20260807 08:41:27.952934 7697 raft_consensus.cc:2454] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520d2aa5686a439a80481b5f3dc68662 in term 1.
I20260807 08:41:27.953135 7795 leader_election.cc:302] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0476cfb332b14d028bb0a82a56ca422b, 520d2aa5686a439a80481b5f3dc68662; no voters:
I20260807 08:41:27.953241 7995 raft_consensus.cc:2781] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:27.953435 7995 raft_consensus.cc:686] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 1 LEADER]: Becoming Leader. State: Replica: 520d2aa5686a439a80481b5f3dc68662, State: Running, Role: LEADER
I20260807 08:41:27.953554 7995 consensus_queue.cc:237] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } }
I20260807 08:41:27.954633 7279 catalog_manager.cc:5495] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 reported cstate change: term changed from 0 to 1, leader changed from <none> to 520d2aa5686a439a80481b5f3dc68662 (127.10.207.3). New cstate: current_term: 1 leader_uuid: "520d2aa5686a439a80481b5f3dc68662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "520d2aa5686a439a80481b5f3dc68662" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 37127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 } health_report { overall_health: UNKNOWN } } }
I20260807 08:41:27.974757 7975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:27.972335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41552 (local address 127.10.207.3:37127)
0807 08:41:27.972409 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:27.972412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:27.972433 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:41:27.972470 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:27.972471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:27.972487 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:27.972580 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:41:27.972732 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.973190 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.973939 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:27.974128 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:27.974207 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:27.974382 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:41:27.974476 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:41:27.974594 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:27.974636 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 08:41:27.974654 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:41:27.976851 7492 raft_consensus.cc:1261] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 1 FOLLOWER]: Refusing update from remote peer 520d2aa5686a439a80481b5f3dc68662: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:27.976851 7697 raft_consensus.cc:1261] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 1 FOLLOWER]: Refusing update from remote peer 520d2aa5686a439a80481b5f3dc68662: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:27.977303 7995 consensus_queue.cc:1035] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fab403ea94174496b01d41c0ba4f0dcc" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 36367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:27.977466 8000 consensus_queue.cc:1035] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0476cfb332b14d028bb0a82a56ca422b" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:27.980521 8007 mvcc.cc:205] Tried to move back new op lower bound from 7315833192350322688 to 7315833192258777088. Current Snapshot: MvccSnapshot[applied={T|T < 7315833192350322688}]
I20260807 08:41:27.981145 8008 mvcc.cc:205] Tried to move back new op lower bound from 7315833192350322688 to 7315833192258777088. Current Snapshot: MvccSnapshot[applied={T|T < 7315833192350322688}]
I20260807 08:41:27.983206 7279 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:27.983314 7279 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:27.984331 7279 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 734d29514f2f4b69a00b3805fe687990: Sending DeleteTablet for 3 replicas of tablet 37096c86161645489aef532b3b116f3a
I20260807 08:41:27.984757 7857 tablet_service.cc:1514] Processing DeleteTablet for tablet 37096c86161645489aef532b3b116f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:27 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:41540
I20260807 08:41:27.984810 7665 tablet_service.cc:1514] Processing DeleteTablet for tablet 37096c86161645489aef532b3b116f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:27 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46944
I20260807 08:41:27.984836 7466 tablet_service.cc:1514] Processing DeleteTablet for tablet 37096c86161645489aef532b3b116f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:27 UTC) from {username='kudu', principal='kudu/127.10.207.62@KRBTEST.COM'} at 127.0.0.1:46190
I20260807 08:41:27.985085 8027 tablet_replica.cc:330] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b: stopping tablet replica
I20260807 08:41:27.985257 8027 raft_consensus.cc:2229] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:27.985285 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 7372
I20260807 08:41:27.985373 8027 raft_consensus.cc:2258] T 37096c86161645489aef532b3b116f3a P 0476cfb332b14d028bb0a82a56ca422b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:27.985993 8026 tablet_replica.cc:330] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: stopping tablet replica
I20260807 08:41:27.986121 8026 raft_consensus.cc:2229] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:27.986255 8026 raft_consensus.cc:2258] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:27.990483 8026 ts_tablet_manager.cc:1905] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:27.992519 8026 ts_tablet_manager.cc:1918] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:27.992666 8026 log.cc:1196] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/37096c86161645489aef532b3b116f3a
I20260807 08:41:27.993046 8026 ts_tablet_manager.cc:1939] T 37096c86161645489aef532b3b116f3a P fab403ea94174496b01d41c0ba4f0dcc: Deleting consensus metadata
I20260807 08:41:27.993638 7259 catalog_manager.cc:4841] TS fab403ea94174496b01d41c0ba4f0dcc (127.10.207.2:36367): tablet 37096c86161645489aef532b3b116f3a (table test-table [id=36542faabf8d4363895661a844bf01ac]) successfully deleted
I20260807 08:41:28.012436 8022 tablet_replica.cc:330] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: stopping tablet replica
I20260807 08:41:28.012671 8022 raft_consensus.cc:2229] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:41:28.012897 8022 raft_consensus.cc:2258] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:28.013441 8022 ts_tablet_manager.cc:1905] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:28.015643 8022 ts_tablet_manager.cc:1918] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:28.015794 8022 log.cc:1196] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/37096c86161645489aef532b3b116f3a
I20260807 08:41:28.016153 8022 ts_tablet_manager.cc:1939] T 37096c86161645489aef532b3b116f3a P 520d2aa5686a439a80481b5f3dc68662: Deleting consensus metadata
I20260807 08:41:28.016772 7261 catalog_manager.cc:4841] TS 520d2aa5686a439a80481b5f3dc68662 (127.10.207.3:37127): tablet 37096c86161645489aef532b3b116f3a (table test-table [id=36542faabf8d4363895661a844bf01ac]) successfully deleted
W20260807 08:41:28.018685 7259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:45183 (error 108)
I20260807 08:41:28.019079 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 7572
I20260807 08:41:28.029671 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.019387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:45183 (local address 127.0.0.1:46196)
0807 08:41:28.029505 (+ 10118us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.029609 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10073}
W20260807 08:41:28.045290 7259 catalog_manager.cc:4584] TS 0476cfb332b14d028bb0a82a56ca422b (127.10.207.1:45183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37096c86161645489aef532b3b116f3a: Network error: Client connection negotiation failed: client connection to 127.10.207.1:45183: connect: Connection refused (error 111)
I20260807 08:41:28.053467 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 7770
I20260807 08:41:28.064168 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 7227
2026-08-07T08:41:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2332 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T08:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:28Z Disabled control of system clock
I20260807 08:41:28.105942 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:40979
--webserver_interface=127.10.207.62
--webserver_port=0
--builtin_ntp_servers=127.10.207.20:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:40979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:41:28.199954 8032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:28.200223 8032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:28.200246 8032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:28.201351 8032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:28.201390 8032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:28.201402 8032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:28.201416 8032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:28.201428 8032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:28.202865 8032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:40979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:40979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:28.203137 8032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:28.203435 8032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:28.209148 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.209456 8039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.209527 8041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:28.209896 8032 server_base.cc:1034] running on GCE node
I20260807 08:41:28.210191 8032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:28.210503 8032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:28.211778 8032 hybrid_clock.cc:648] HybridClock initialized: now 1786092088211744 us; error 58 us; skew 500 ppm
I20260807 08:41:28.213254 8032 webserver.cc:460] Webserver started at http://127.10.207.62:40755/ using document root <none> and password file <none>
I20260807 08:41:28.213517 8032 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:28.213567 8032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:28.213688 8032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:28.214965 8032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "7812f7f3586f4ff09897238842386423"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "610508b07b5b1b67f7bc405905ad5323"
server_key_iv: "e05f11610df557a6f8dc7a0bcb446f30"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.215355 8032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "7812f7f3586f4ff09897238842386423"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "610508b07b5b1b67f7bc405905ad5323"
server_key_iv: "e05f11610df557a6f8dc7a0bcb446f30"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.216739 8032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:28.217566 8054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.217837 8032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:28.217958 8032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "7812f7f3586f4ff09897238842386423"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "610508b07b5b1b67f7bc405905ad5323"
server_key_iv: "e05f11610df557a6f8dc7a0bcb446f30"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.218044 8032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:28.238828 8032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:28.239204 8032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:28.239349 8032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:28.244652 8032 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:40979
I20260807 08:41:28.244823 8131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:40979 every 8 connection(s)
I20260807 08:41:28.245200 8032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:28.246687 8132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:28.249414 8132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423: Bootstrap starting.
I20260807 08:41:28.250029 8132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:28.250344 8132 log.cc:824] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:28.251256 8132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423: No bootstrap required, opened a new log
I20260807 08:41:28.252501 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 8032
I20260807 08:41:28.252614 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:28.252877 11068 external_mini_cluster.cc:1428] Setting key 4b2f229a5171314ddd966a732f877909
I20260807 08:41:28.253243 8132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7812f7f3586f4ff09897238842386423" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40979 } }
I20260807 08:41:28.253448 8132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:28.253491 8132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7812f7f3586f4ff09897238842386423, State: Initialized, Role: FOLLOWER
I20260807 08:41:28.253643 8132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7812f7f3586f4ff09897238842386423" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40979 } }
I20260807 08:41:28.253750 8132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:28.253794 8132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:28.253842 8132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:28.254596 8132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7812f7f3586f4ff09897238842386423" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40979 } }
I20260807 08:41:28.254712 8132 leader_election.cc:302] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7812f7f3586f4ff09897238842386423; no voters:
I20260807 08:41:28.254895 8132 leader_election.cc:288] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:28.254993 8145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:28.255232 8132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:28.255218 8145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [term 1 LEADER]: Becoming Leader. State: Replica: 7812f7f3586f4ff09897238842386423, State: Running, Role: LEADER
I20260807 08:41:28.255422 8145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7812f7f3586f4ff09897238842386423" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40979 } }
I20260807 08:41:28.255937 8147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7812f7f3586f4ff09897238842386423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7812f7f3586f4ff09897238842386423" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40979 } } }
I20260807 08:41:28.256043 8147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:28.256227 8147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7812f7f3586f4ff09897238842386423. Latest consensus state: current_term: 1 leader_uuid: "7812f7f3586f4ff09897238842386423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7812f7f3586f4ff09897238842386423" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40979 } } }
I20260807 08:41:28.256294 8147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:28.256383 8158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:28.257092 8158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:28.258919 8158 catalog_manager.cc:1347] Generated new cluster ID: 2f7c5fd916a14d01aa56aadf50efd637
I20260807 08:41:28.259402 8158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:28.263752 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.253594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58286 (local address 127.10.207.62:40979)
0807 08:41:28.253957 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.253966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.254004 (+ 38us) server_negotiation.cc:407] Connection header received
0807 08:41:28.254133 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.254140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.254218 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.254382 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:28.254639 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.256550 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.257561 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.258502 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.258690 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.262905 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:28.262910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:28.262911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:28.263168 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:28.263213 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:28.263227 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:28.263314 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.263406 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.263511 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":140,"threads_started":1}
I20260807 08:41:28.282677 8158 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:28.283557 8158 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:28.291090 8158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423: Generated new TSK 0
I20260807 08:41:28.291306 8158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:28.298763 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.1:0
--local_ip_for_outbound_sockets=127.10.207.1
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40979
--builtin_ntp_servers=127.10.207.20:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:41:28.412634 8174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:28.412885 8174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:28.412909 8174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:28.414194 8174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:28.414243 8174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:28.414270 8174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.1
I20260807 08:41:28.415792 8174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.207.1
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.1
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:28.416088 8174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:28.416462 8174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:28.422232 8181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.422286 8182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.422236 8187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:28.422840 8174 server_base.cc:1034] running on GCE node
I20260807 08:41:28.423173 8174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:28.423483 8174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:28.424693 8174 hybrid_clock.cc:648] HybridClock initialized: now 1786092088424664 us; error 49 us; skew 500 ppm
I20260807 08:41:28.426397 8174 webserver.cc:460] Webserver started at http://127.10.207.1:43625/ using document root <none> and password file <none>
I20260807 08:41:28.426760 8174 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:28.426865 8174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:28.427057 8174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:28.428303 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data/instance:
uuid: "2b4652bd53b4472f9c1ad352e44d28e1"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "6bb8ada2b32209df0ecc769160711935"
server_key_iv: "9e9ebec49652d227774bd0603400e601"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.428720 8174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance:
uuid: "2b4652bd53b4472f9c1ad352e44d28e1"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "6bb8ada2b32209df0ecc769160711935"
server_key_iv: "9e9ebec49652d227774bd0603400e601"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.430341 8174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:28.431447 8194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.431746 8174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:28.431893 8174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/wal
uuid: "2b4652bd53b4472f9c1ad352e44d28e1"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "6bb8ada2b32209df0ecc769160711935"
server_key_iv: "9e9ebec49652d227774bd0603400e601"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.431983 8174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:28.455931 8174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:28.456414 8174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:28.456606 8174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:28.456916 8174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:28.457401 8174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:28.457473 8174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.457520 8174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:28.457546 8174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.465318 8174 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.1:35947
I20260807 08:41:28.465378 8380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.1:35947 every 8 connection(s)
I20260807 08:41:28.465777 8174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/data/info.pb
I20260807 08:41:28.467636 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 8174
I20260807 08:41:28.467734 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-0/wal/instance
I20260807 08:41:28.467957 11068 external_mini_cluster.cc:1428] Setting key 41928788990823f524e65cbb4a5b331f
I20260807 08:41:28.469079 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.2:0
--local_ip_for_outbound_sockets=127.10.207.2
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40979
--builtin_ntp_servers=127.10.207.20:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:41:28.473325 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.466698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.1:51535 (local address 127.10.207.62:40979)
0807 08:41:28.466825 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.466866 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.467165 (+ 299us) server_negotiation.cc:407] Connection header received
0807 08:41:28.467967 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.467969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.467985 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.468057 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:28.469348 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.469958 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.471303 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.471542 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.471628 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.472856 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:28.472858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:28.472858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:28.472867 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:28.472888 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:28.472893 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:28.472947 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.473088 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.473163 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 08:41:28.473464 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.466702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:40979 (local address 127.10.207.1:51535)
0807 08:41:28.466999 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.467048 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.467680 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.468263 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.468267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.468277 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:28.469107 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.469112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.470087 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.470088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.471129 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.471134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.471767 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.471769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.471948 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.472657 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:28.472658 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:28.472695 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:28.472742 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:28.472972 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:28.472979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:28.472982 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.473070 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.473188 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":108,"threads_started":1}
I20260807 08:41:28.475093 8383 heartbeater.cc:347] Connected to a master server at 127.10.207.62:40979
I20260807 08:41:28.475222 8383 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:28.475445 8383 heartbeater.cc:511] Master 127.10.207.62:40979 requested a full tablet report, sending...
I20260807 08:41:28.476054 8078 ts_manager.cc:194] Registered new tserver with Master: 2b4652bd53b4472f9c1ad352e44d28e1 (127.10.207.1:35947)
I20260807 08:41:28.476740 8078 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.1:51535
W20260807 08:41:28.560974 8390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:28.561218 8390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:28.561259 8390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:28.562376 8390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:28.562441 8390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:28.562485 8390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.2
I20260807 08:41:28.563884 8390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.207.2
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.2
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:28.564420 8390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:28.564735 8390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:28.570276 8396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.570339 8397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.570310 8400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:28.570703 8390 server_base.cc:1034] running on GCE node
I20260807 08:41:28.570919 8390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:28.571172 8390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:28.572350 8390 hybrid_clock.cc:648] HybridClock initialized: now 1786092088572332 us; error 40 us; skew 500 ppm
I20260807 08:41:28.573808 8390 webserver.cc:460] Webserver started at http://127.10.207.2:46153/ using document root <none> and password file <none>
I20260807 08:41:28.574175 8390 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:28.574261 8390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:28.574426 8390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:28.575677 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data/instance:
uuid: "39b6d53f909f4706b9060f03a5e697ac"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "53bdcfff11d36f228b70766c68e28ba8"
server_key_iv: "c0bead477fb1dfe51e5ef8f47d0c733f"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.576278 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance:
uuid: "39b6d53f909f4706b9060f03a5e697ac"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "53bdcfff11d36f228b70766c68e28ba8"
server_key_iv: "c0bead477fb1dfe51e5ef8f47d0c733f"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.577883 8390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:28.578954 8409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.579221 8390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:28.579356 8390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal
uuid: "39b6d53f909f4706b9060f03a5e697ac"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "53bdcfff11d36f228b70766c68e28ba8"
server_key_iv: "c0bead477fb1dfe51e5ef8f47d0c733f"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.579452 8390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:28.592275 8390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:28.592586 8390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:28.592746 8390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:28.592985 8390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:28.593336 8390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:28.593402 8390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.593438 8390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:28.593469 8390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.600818 8390 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.2:39147
I20260807 08:41:28.600981 8584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.2:39147 every 8 connection(s)
I20260807 08:41:28.601279 8390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/data/info.pb
I20260807 08:41:28.606700 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 8390
I20260807 08:41:28.606822 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal/instance
I20260807 08:41:28.607024 11068 external_mini_cluster.cc:1428] Setting key 7997e5d53bf94508a15a5c4642c8a182
I20260807 08:41:28.607932 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.602217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.2:43419 (local address 127.10.207.62:40979)
0807 08:41:28.602282 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.602284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.602577 (+ 293us) server_negotiation.cc:407] Connection header received
0807 08:41:28.603137 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.603138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.603155 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.603213 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:28.603891 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.604387 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.605572 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.605770 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.605857 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.607522 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:28.607523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:28.607523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:28.607528 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:28.607541 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:28.607545 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:28.607642 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.607755 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.607813 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:41:28.608093 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.602175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:40979 (local address 127.10.207.2:43419)
0807 08:41:28.602429 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.602464 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.603025 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.603281 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.603286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.603295 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:28.603788 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.603792 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.604490 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.604491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.605421 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.605425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.605871 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.605872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.606016 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.607310 (+ 1294us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:28.607311 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:28.607348 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:28.607376 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:28.607645 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:28.607652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:28.607655 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.607752 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.607853 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":117,"threads_started":1}
I20260807 08:41:28.608687 8586 heartbeater.cc:347] Connected to a master server at 127.10.207.62:40979
I20260807 08:41:28.608774 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.207.3:0
--local_ip_for_outbound_sockets=127.10.207.3
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40979
--builtin_ntp_servers=127.10.207.20:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:41:28.608821 8586 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:28.609062 8586 heartbeater.cc:511] Master 127.10.207.62:40979 requested a full tablet report, sending...
I20260807 08:41:28.609511 8078 ts_manager.cc:194] Registered new tserver with Master: 39b6d53f909f4706b9060f03a5e697ac (127.10.207.2:39147)
I20260807 08:41:28.609899 8078 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.2:43419
W20260807 08:41:28.701195 8592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:28.701435 8592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:28.701457 8592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:28.702705 8592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:28.702751 8592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:28.702777 8592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.207.3
I20260807 08:41:28.704176 8592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.207.3
--webserver_port=0
--tserver_master_addrs=127.10.207.62:40979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.207.3
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:28.704468 8592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:28.704836 8592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:28.710556 8604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.710552 8602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:28.710839 8601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:28.710718 8592 server_base.cc:1034] running on GCE node
I20260807 08:41:28.711292 8592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:28.711613 8592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:28.712780 8592 hybrid_clock.cc:648] HybridClock initialized: now 1786092088712759 us; error 43 us; skew 500 ppm
I20260807 08:41:28.714226 8592 webserver.cc:460] Webserver started at http://127.10.207.3:39063/ using document root <none> and password file <none>
I20260807 08:41:28.714525 8592 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:28.714658 8592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:28.714826 8592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:28.716079 8592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data/instance:
uuid: "cb48893f59f146d0b32e82423ebb01fd"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "9e1d84c9acf00050395d2423c210ee9e"
server_key_iv: "1c706da7e186000539288a31c791fcf2"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.716502 8592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance:
uuid: "cb48893f59f146d0b32e82423ebb01fd"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "9e1d84c9acf00050395d2423c210ee9e"
server_key_iv: "1c706da7e186000539288a31c791fcf2"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.718119 8592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:28.719164 8611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.719432 8592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:41:28.719550 8592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal
uuid: "cb48893f59f146d0b32e82423ebb01fd"
format_stamp: "Formatted at 2026-08-07 08:41:28 on dist-test-slave-h6n0"
server_key: "9e1d84c9acf00050395d2423c210ee9e"
server_key_iv: "1c706da7e186000539288a31c791fcf2"
server_key_version: "encryptionkey@0"
I20260807 08:41:28.719638 8592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:28.730938 8592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:28.731235 8592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:28.731385 8592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:28.731608 8592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:41:28.731952 8592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:41:28.732018 8592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.732069 8592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:41:28.732097 8592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:28.738781 8592 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.3:43687
I20260807 08:41:28.738891 8784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.3:43687 every 8 connection(s)
I20260807 08:41:28.739209 8592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/data/info.pb
I20260807 08:41:28.745519 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.740160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:57999 (local address 127.10.207.62:40979)
0807 08:41:28.740329 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.740332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.740557 (+ 225us) server_negotiation.cc:407] Connection header received
0807 08:41:28.741372 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.741375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.741391 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.741461 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:28.742248 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.742783 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.743954 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.744151 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.744246 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.745201 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:28.745202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:28.745203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:28.745208 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:28.745229 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:28.745233 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:28.745291 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.745409 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.745438 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 08:41:28.745747 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.740091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:40979 (local address 127.10.207.3:57999)
0807 08:41:28.740407 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.740455 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.741093 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.741546 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.741551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.741569 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:28.742129 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.742135 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.742891 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.742892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.743824 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.743828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.744259 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.744260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.744397 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.745004 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:41:28.745005 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:28.745033 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:41:28.745056 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:28.745306 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:28.745312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:28.745315 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.745397 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.745500 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":127,"threads_started":1}
I20260807 08:41:28.746057 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 8592
I20260807 08:41:28.746217 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal/instance
I20260807 08:41:28.746457 8785 heartbeater.cc:347] Connected to a master server at 127.10.207.62:40979
I20260807 08:41:28.746528 11068 external_mini_cluster.cc:1428] Setting key b437aee386da2a7a13770e09e83ac4b4
I20260807 08:41:28.746634 8785 heartbeater.cc:464] Registering TS with master...
I20260807 08:41:28.746917 8785 heartbeater.cc:511] Master 127.10.207.62:40979 requested a full tablet report, sending...
I20260807 08:41:28.747432 8075 ts_manager.cc:194] Registered new tserver with Master: cb48893f59f146d0b32e82423ebb01fd (127.10.207.3:43687)
I20260807 08:41:28.747895 8075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.207.3:57999
I20260807 08:41:28.758857 11068 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:41:28.763996 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.760692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.10.207.62:40979)
0807 08:41:28.760746 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.760748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.760853 (+ 105us) server_negotiation.cc:407] Connection header received
0807 08:41:28.760908 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.760908 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.760917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.760970 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:28.761171 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.761620 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.762490 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.762749 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.762821 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.763668 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:28.763670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:28.763670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:28.763675 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:28.763690 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:28.763694 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:28.763764 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.763866 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.763887 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:41:28.766156 8075 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:41:28.767814 8075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:41:28.775328 8798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.770735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:39147 (local address 127.0.0.1:35458)
0807 08:41:28.770866 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.770908 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.771008 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.771641 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.771645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.771649 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:28.771732 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.771734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.772782 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.772783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.774101 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.774104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.774960 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.774961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.775068 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.775086 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.775187 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.775259 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260807 08:41:28.775341 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.770923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35458 (local address 127.10.207.2:39147)
0807 08:41:28.771433 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.771437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.771449 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:41:28.771496 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.771498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.771516 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.771834 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:28.771907 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.772688 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.774205 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.774850 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.774948 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.775043 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.775208 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.775251 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":102,"threads_started":1}
I20260807 08:41:28.775635 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.770780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:35947 (local address 127.0.0.1:39166)
0807 08:41:28.771387 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.771398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.771486 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.771730 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.771731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.771734 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:28.771793 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.771795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.773146 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.773147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.774252 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.774255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.775337 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.775337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.775421 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.775426 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.775489 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.775549 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":32,"thread_start_us":85,"threads_started":1}
I20260807 08:41:28.775974 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.771141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39166 (local address 127.10.207.1:35947)
0807 08:41:28.771310 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.771316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.771473 (+ 157us) server_negotiation.cc:407] Connection header received
0807 08:41:28.771558 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.771561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.771580 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.771661 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:28.771871 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.773041 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.774363 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.775131 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.775199 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.775293 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.775829 (+ 536us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.775882 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":112,"threads_started":1}
I20260807 08:41:28.776043 8481 tablet_service.cc:1467] Processing CreateTablet for tablet 915579dcaa2c42af9c7a0659a1f3c6e9 (DEFAULT_TABLE table=test-table [id=f669a125bd7543fd8422a409ae486797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:28.776363 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.771133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.10.207.3:43687)
0807 08:41:28.771338 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.771343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.771687 (+ 344us) server_negotiation.cc:407] Connection header received
0807 08:41:28.771734 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.771736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.771751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.771834 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:28.772002 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.773078 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.774312 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.775122 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.775199 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.775293 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.776192 (+ 899us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.776236 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":128,"threads_started":1}
I20260807 08:41:28.776505 8481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915579dcaa2c42af9c7a0659a1f3c6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:28.776652 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.771001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.3:43687 (local address 127.0.0.1:53014)
0807 08:41:28.771566 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.771575 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.771621 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.771854 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.771855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.771857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:28.771906 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.771907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.773182 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.773183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.774208 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.774211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.775333 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.775334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.775958 (+ 624us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.775966 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.776503 (+ 537us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.776563 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":50,"threads_started":1}
I20260807 08:41:28.776651 8273 tablet_service.cc:1467] Processing CreateTablet for tablet 915579dcaa2c42af9c7a0659a1f3c6e9 (DEFAULT_TABLE table=test-table [id=f669a125bd7543fd8422a409ae486797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:28.777206 8273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915579dcaa2c42af9c7a0659a1f3c6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:28.777477 8676 tablet_service.cc:1467] Processing CreateTablet for tablet 915579dcaa2c42af9c7a0659a1f3c6e9 (DEFAULT_TABLE table=test-table [id=f669a125bd7543fd8422a409ae486797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:41:28.777849 8676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915579dcaa2c42af9c7a0659a1f3c6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:28.780737 8810 tablet_bootstrap.cc:492] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Bootstrap starting.
I20260807 08:41:28.781127 8814 tablet_bootstrap.cc:492] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1: Bootstrap starting.
I20260807 08:41:28.781267 8811 tablet_bootstrap.cc:492] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Bootstrap starting.
I20260807 08:41:28.781774 8810 tablet_bootstrap.cc:654] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:28.781894 8814 tablet_bootstrap.cc:654] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:28.781908 8811 tablet_bootstrap.cc:654] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:28.782173 8810 log.cc:824] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:28.782272 8811 log.cc:824] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:28.782272 8814 log.cc:824] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:28.783077 8810 tablet_bootstrap.cc:492] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: No bootstrap required, opened a new log
I20260807 08:41:28.783169 8814 tablet_bootstrap.cc:492] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1: No bootstrap required, opened a new log
I20260807 08:41:28.783174 8810 ts_tablet_manager.cc:1397] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:41:28.783255 8814 ts_tablet_manager.cc:1397] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:41:28.785044 8814 raft_consensus.cc:348] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.785176 8814 raft_consensus.cc:374] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:28.785198 8814 raft_consensus.cc:729] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4652bd53b4472f9c1ad352e44d28e1, State: Initialized, Role: FOLLOWER
I20260807 08:41:28.785315 8814 consensus_queue.cc:260] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.785584 8814 ts_tablet_manager.cc:1428] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:41:28.785615 8383 heartbeater.cc:503] Master 127.10.207.62:40979 was elected leader, sending a full tablet report...
I20260807 08:41:28.785796 8810 raft_consensus.cc:348] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.786017 8810 raft_consensus.cc:374] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:28.786077 8810 raft_consensus.cc:729] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb48893f59f146d0b32e82423ebb01fd, State: Initialized, Role: FOLLOWER
I20260807 08:41:28.786231 8810 consensus_queue.cc:260] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.786569 8810 ts_tablet_manager.cc:1428] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 08:41:28.786669 8785 heartbeater.cc:503] Master 127.10.207.62:40979 was elected leader, sending a full tablet report...
I20260807 08:41:28.787483 8811 tablet_bootstrap.cc:492] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: No bootstrap required, opened a new log
I20260807 08:41:28.787575 8811 ts_tablet_manager.cc:1397] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 08:41:28.789593 8820 raft_consensus.cc:482] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:41:28.789708 8820 raft_consensus.cc:504] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.789775 8811 raft_consensus.cc:348] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.789951 8811 raft_consensus.cc:374] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:28.789987 8811 raft_consensus.cc:729] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b6d53f909f4706b9060f03a5e697ac, State: Initialized, Role: FOLLOWER
I20260807 08:41:28.790112 8820 leader_election.cc:288] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b6d53f909f4706b9060f03a5e697ac (127.10.207.2:39147), 2b4652bd53b4472f9c1ad352e44d28e1 (127.10.207.1:35947)
I20260807 08:41:28.790115 8811 consensus_queue.cc:260] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.790414 8811 ts_tablet_manager.cc:1428] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:41:28.790608 8586 heartbeater.cc:503] Master 127.10.207.62:40979 was elected leader, sending a full tablet report...
I20260807 08:41:28.794205 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.790564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:58337 (local address 127.10.207.2:39147)
0807 08:41:28.790619 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.790622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.790634 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:41:28.790666 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.790667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.790702 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.790767 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:28.790952 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.791677 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.793230 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.793803 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.793899 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.793905 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.794101 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.794141 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:41:28.794214 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.790334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.207.3:39777 (local address 127.10.207.1:35947)
0807 08:41:28.790387 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.790390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.790732 (+ 342us) server_negotiation.cc:407] Connection header received
0807 08:41:28.790844 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.790845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.790858 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.790932 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:41:28.791201 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.791880 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.793230 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.793815 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.793896 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.793905 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.794101 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.794142 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:41:28.794230 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.790238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.2:39147 (local address 127.10.207.3:58337)
0807 08:41:28.790397 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.790408 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.790449 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.790779 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.790780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.790782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:28.790852 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.790855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.791758 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.791759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.793092 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.793095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.793928 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.793929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.794007 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.794012 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.794077 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.794167 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260807 08:41:28.794230 8822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.790258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:35947 (local address 127.10.207.3:39777)
0807 08:41:28.790641 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.790655 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:28.790762 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:28.791015 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:28.791016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:28.791018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:41:28.791100 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.791102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.791961 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.791962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.793092 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:28.793095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.793918 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.793920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:28.794010 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.794012 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:41:28.794077 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:41:28.794125 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":5,"thread_start_us":97,"threads_started":1}
I20260807 08:41:28.794683 8516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915579dcaa2c42af9c7a0659a1f3c6e9" candidate_uuid: "cb48893f59f146d0b32e82423ebb01fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b6d53f909f4706b9060f03a5e697ac" is_pre_election: true
I20260807 08:41:28.794689 8304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915579dcaa2c42af9c7a0659a1f3c6e9" candidate_uuid: "cb48893f59f146d0b32e82423ebb01fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" is_pre_election: true
I20260807 08:41:28.794844 8304 raft_consensus.cc:2454] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb48893f59f146d0b32e82423ebb01fd in term 0.
I20260807 08:41:28.794844 8516 raft_consensus.cc:2454] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb48893f59f146d0b32e82423ebb01fd in term 0.
I20260807 08:41:28.795133 8616 leader_election.cc:302] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b4652bd53b4472f9c1ad352e44d28e1, cb48893f59f146d0b32e82423ebb01fd; no voters:
I20260807 08:41:28.795302 8820 raft_consensus.cc:2781] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:41:28.795398 8820 raft_consensus.cc:482] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:41:28.795429 8820 raft_consensus.cc:3037] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:28.796165 8820 raft_consensus.cc:504] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.796329 8820 leader_election.cc:288] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [CANDIDATE]: Term 1 election: Requested vote from peers 39b6d53f909f4706b9060f03a5e697ac (127.10.207.2:39147), 2b4652bd53b4472f9c1ad352e44d28e1 (127.10.207.1:35947)
I20260807 08:41:28.796482 8516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915579dcaa2c42af9c7a0659a1f3c6e9" candidate_uuid: "cb48893f59f146d0b32e82423ebb01fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b6d53f909f4706b9060f03a5e697ac"
I20260807 08:41:28.796553 8516 raft_consensus.cc:3037] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:28.796540 8304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915579dcaa2c42af9c7a0659a1f3c6e9" candidate_uuid: "cb48893f59f146d0b32e82423ebb01fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b4652bd53b4472f9c1ad352e44d28e1"
I20260807 08:41:28.796622 8304 raft_consensus.cc:3037] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:28.797420 8516 raft_consensus.cc:2454] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb48893f59f146d0b32e82423ebb01fd in term 1.
I20260807 08:41:28.797425 8304 raft_consensus.cc:2454] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb48893f59f146d0b32e82423ebb01fd in term 1.
I20260807 08:41:28.797626 8616 leader_election.cc:302] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b4652bd53b4472f9c1ad352e44d28e1, cb48893f59f146d0b32e82423ebb01fd; no voters:
I20260807 08:41:28.797737 8820 raft_consensus.cc:2781] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:28.797902 8820 raft_consensus.cc:686] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 1 LEADER]: Becoming Leader. State: Replica: cb48893f59f146d0b32e82423ebb01fd, State: Running, Role: LEADER
I20260807 08:41:28.798007 8820 consensus_queue.cc:237] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } }
I20260807 08:41:28.798849 8075 catalog_manager.cc:5495] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd reported cstate change: term changed from 0 to 1, leader changed from <none> to cb48893f59f146d0b32e82423ebb01fd (127.10.207.3). New cstate: current_term: 1 leader_uuid: "cb48893f59f146d0b32e82423ebb01fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb48893f59f146d0b32e82423ebb01fd" member_type: VOTER last_known_addr { host: "127.10.207.3" port: 43687 } health_report { overall_health: HEALTHY } } }
I20260807 08:41:28.808157 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.805702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.10.207.3:43687)
0807 08:41:28.805806 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:28.805808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:28.805816 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:41:28.805847 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:28.805848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:28.805872 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:28.805952 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:41:28.806093 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.806598 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.807353 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:28.807568 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:28.807645 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:28.807808 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:41:28.807914 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:41:28.808012 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:28.808031 (+ 19us) server_negotiation.cc:299] Negotiation successful
0807 08:41:28.808060 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 08:41:28.810477 8516 raft_consensus.cc:1261] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 1 FOLLOWER]: Refusing update from remote peer cb48893f59f146d0b32e82423ebb01fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:28.810609 8304 raft_consensus.cc:1261] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 1 FOLLOWER]: Refusing update from remote peer cb48893f59f146d0b32e82423ebb01fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:28.810858 8823 consensus_queue.cc:1035] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b6d53f909f4706b9060f03a5e697ac" member_type: VOTER last_known_addr { host: "127.10.207.2" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:28.810989 8820 consensus_queue.cc:1035] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b4652bd53b4472f9c1ad352e44d28e1" member_type: VOTER last_known_addr { host: "127.10.207.1" port: 35947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:28.813359 8830 mvcc.cc:205] Tried to move back new op lower bound from 7315833195764113408 to 7315833195717472256. Current Snapshot: MvccSnapshot[applied={T|T < 7315833195764113408}]
I20260807 08:41:28.814687 8831 mvcc.cc:205] Tried to move back new op lower bound from 7315833195764113408 to 7315833195717472256. Current Snapshot: MvccSnapshot[applied={T|T < 7315833195764113408}]
I20260807 08:41:28.814728 8833 mvcc.cc:205] Tried to move back new op lower bound from 7315833195764113408 to 7315833195717472256. Current Snapshot: MvccSnapshot[applied={T|T < 7315833195764113408}]
I20260807 08:41:28.816244 8075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:28.816349 8075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:41:28.817310 8075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7812f7f3586f4ff09897238842386423: Sending DeleteTablet for 3 replicas of tablet 915579dcaa2c42af9c7a0659a1f3c6e9
I20260807 08:41:28.817706 8481 tablet_service.cc:1514] Processing DeleteTablet for tablet 915579dcaa2c42af9c7a0659a1f3c6e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:28 UTC) from {username='slave'} at 127.0.0.1:35458
I20260807 08:41:28.817775 8676 tablet_service.cc:1514] Processing DeleteTablet for tablet 915579dcaa2c42af9c7a0659a1f3c6e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:28 UTC) from {username='slave'} at 127.0.0.1:53014
I20260807 08:41:28.817790 8273 tablet_service.cc:1514] Processing DeleteTablet for tablet 915579dcaa2c42af9c7a0659a1f3c6e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:41:28 UTC) from {username='slave'} at 127.0.0.1:39166
I20260807 08:41:28.818005 8844 tablet_replica.cc:330] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: stopping tablet replica
I20260807 08:41:28.818127 8844 raft_consensus.cc:2229] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:41:28.818264 8844 raft_consensus.cc:2258] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:28.818326 8845 tablet_replica.cc:330] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1: stopping tablet replica
I20260807 08:41:28.818419 8845 raft_consensus.cc:2229] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 2b4652bd53b4472f9c1ad352e44d28e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:28.818432 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 8174
I20260807 08:41:28.823721 8841 tablet_replica.cc:330] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: stopping tablet replica
I20260807 08:41:28.823851 8841 raft_consensus.cc:2229] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:41:28.823969 8841 raft_consensus.cc:2258] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:41:28.827787 8841 ts_tablet_manager.cc:1905] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:28.829483 8844 ts_tablet_manager.cc:1905] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:41:28.829869 8841 ts_tablet_manager.cc:1918] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:28.829941 8841 log.cc:1196] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-1/wal/wals/915579dcaa2c42af9c7a0659a1f3c6e9
I20260807 08:41:28.830212 8841 ts_tablet_manager.cc:1939] T 915579dcaa2c42af9c7a0659a1f3c6e9 P 39b6d53f909f4706b9060f03a5e697ac: Deleting consensus metadata
I20260807 08:41:28.830747 8057 catalog_manager.cc:4841] TS 39b6d53f909f4706b9060f03a5e697ac (127.10.207.2:39147): tablet 915579dcaa2c42af9c7a0659a1f3c6e9 (table test-table [id=f669a125bd7543fd8422a409ae486797]) successfully deleted
I20260807 08:41:28.831357 8844 ts_tablet_manager.cc:1918] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:41:28.831465 8844 log.cc:1196] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Deleting WAL directory at /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091830136701-11068-0/minicluster-data/ts-2/wal/wals/915579dcaa2c42af9c7a0659a1f3c6e9
I20260807 08:41:28.831964 8844 ts_tablet_manager.cc:1939] T 915579dcaa2c42af9c7a0659a1f3c6e9 P cb48893f59f146d0b32e82423ebb01fd: Deleting consensus metadata
I20260807 08:41:28.832455 8058 catalog_manager.cc:4841] TS cb48893f59f146d0b32e82423ebb01fd (127.10.207.3:43687): tablet 915579dcaa2c42af9c7a0659a1f3c6e9 (table test-table [id=f669a125bd7543fd8422a409ae486797]) successfully deleted
W20260807 08:41:28.837625 8058 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.207.1:35947 (error 108)
I20260807 08:41:28.838303 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:28.838132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.1:35947 (local address 127.0.0.1:39168)
0807 08:41:28.838209 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:28.838258 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.1:35947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:41:28.838405 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 8390
W20260807 08:41:28.839109 8058 catalog_manager.cc:4584] TS 2b4652bd53b4472f9c1ad352e44d28e1 (127.10.207.1:35947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 915579dcaa2c42af9c7a0659a1f3c6e9: Network error: Client connection negotiation failed: client connection to 127.10.207.1:35947: connect: Connection refused (error 111)
I20260807 08:41:28.855528 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 8592
I20260807 08:41:28.872066 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 8032
2026-08-07T08:41:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (807 ms)
[----------] 6 tests from EncryptionPolicyTest (9610 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:28 dist-test-slave-h6n0 krb5kdc[8849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:28 dist-test-slave-h6n0 krb5kdc[8849](info): set up 2 sockets
Aug 07 08:41:28 dist-test-slave-h6n0 krb5kdc[8849](info): commencing operation
krb5kdc: starting...
W20260807 08:41:30.146755 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:30 dist-test-slave-h6n0 krb5kdc[8849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:30.283002 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42933
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:30.376425 8867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:30.376621 8867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:30.376643 8867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:30.377745 8867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:30.377789 8867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:30.377803 8867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:30.377816 8867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:30.377830 8867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:30.379359 8867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42933
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:30.379595 8867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:30.379851 8867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:30.385044 8876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:30.385084 8875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:30.385049 8878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:30.385272 8867 server_base.cc:1034] running on GCE node
I20260807 08:41:30.385527 8867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:30.385766 8867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:30.386907 8867 hybrid_clock.cc:648] HybridClock initialized: now 1786092090386894 us; error 39 us; skew 500 ppm
Aug 07 08:41:30 dist-test-slave-h6n0 krb5kdc[8849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092090, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:30.389536 8867 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:30.389978 8867 webserver.cc:460] Webserver started at http://127.0.0.1:38943/ using document root <none> and password file <none>
I20260807 08:41:30.390228 8867 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:30.390295 8867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:30.390442 8867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:30.391445 8867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad"
format_stamp: "Formatted at 2026-08-07 08:41:30 on dist-test-slave-h6n0"
server_key: "253356bf904ccb3e4ca1824ba9de9350"
server_key_iv: "a2f5a944aeb2234426df0ca35bf0e365"
server_key_version: "encryptionkey@0"
I20260807 08:41:30.391821 8867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad"
format_stamp: "Formatted at 2026-08-07 08:41:30 on dist-test-slave-h6n0"
server_key: "253356bf904ccb3e4ca1824ba9de9350"
server_key_iv: "a2f5a944aeb2234426df0ca35bf0e365"
server_key_version: "encryptionkey@0"
I20260807 08:41:30.393190 8867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:30.393987 8887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:30.394222 8867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:30.394330 8867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad"
format_stamp: "Formatted at 2026-08-07 08:41:30 on dist-test-slave-h6n0"
server_key: "253356bf904ccb3e4ca1824ba9de9350"
server_key_iv: "a2f5a944aeb2234426df0ca35bf0e365"
server_key_version: "encryptionkey@0"
I20260807 08:41:30.394407 8867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:30.419373 8867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:30.421519 8867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:30.421681 8867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:30.426467 8867 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42933
I20260807 08:41:30.426585 8979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42933 every 8 connection(s)
I20260807 08:41:30.426936 8867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:30.428023 8982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:30.430809 8982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad: Bootstrap starting.
I20260807 08:41:30.431628 8982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:30.431996 8982 log.cc:824] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:30.432992 8982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad: No bootstrap required, opened a new log
I20260807 08:41:30.433712 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 8867
I20260807 08:41:30.433880 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:30.434103 11068 external_mini_cluster.cc:1428] Setting key 0f197c95ba66e114668ba86183f4b97a
I20260807 08:41:30.435622 8982 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260807 08:41:30.435835 8982 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:30.435882 8982 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e2fe2da2d2495fada2e33d4d19e9ad, State: Initialized, Role: FOLLOWER
I20260807 08:41:30.436029 8982 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260807 08:41:30.436120 8982 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:30.436146 8982 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:30.436219 8982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:30.437623 8982 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260807 08:41:30.437749 8982 leader_election.cc:302] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9e2fe2da2d2495fada2e33d4d19e9ad; no voters:
I20260807 08:41:30.437922 8982 leader_election.cc:288] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:30.438033 8988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:30.438218 8982 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:30.438292 8988 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [term 1 LEADER]: Becoming Leader. State: Replica: a9e2fe2da2d2495fada2e33d4d19e9ad, State: Running, Role: LEADER
I20260807 08:41:30.438416 8988 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260807 08:41:30.439448 8989 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } } }
I20260807 08:41:30.439567 8989 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:30.440086 8990 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9e2fe2da2d2495fada2e33d4d19e9ad. Latest consensus state: current_term: 1 leader_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e2fe2da2d2495fada2e33d4d19e9ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } } }
I20260807 08:41:30.440187 8990 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:30.440307 8999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:30.440935 8999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:41:30 dist-test-slave-h6n0 krb5kdc[8849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:41:30.443964 8999 catalog_manager.cc:1347] Generated new cluster ID: ee4d0dac1db343f49f3dac9e1a8aa8a6
I20260807 08:41:30.444051 8999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:30.448487 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:30.434852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55847 (local address 127.0.0.1:42933)
0807 08:41:30.435108 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:30.435121 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:30.435162 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:41:30.435563 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:30.435573 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:30.435646 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:30.435818 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:30.436721 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:30.438239 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:30.439515 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:30.440322 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:30.440442 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:30.443914 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:30.443919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:30.443919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:30.443933 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:30.446127 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:30.446844 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:30.446847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:30.446848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:30.446989 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:30.447274 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:30.447275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:30.447276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:30.447597 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:30.447777 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:30.448001 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 08:41:30.448267 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
I20260807 08:41:30.449704 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:30.455709 8999 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:30.456641 8999 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:30.459756 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:30.451536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41565 (local address 127.0.0.1:42933)
0807 08:41:30.451581 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:30.451584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:30.452001 (+ 417us) server_negotiation.cc:407] Connection header received
0807 08:41:30.452054 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:30.452055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:30.452067 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:30.452131 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:30.452782 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:30.453590 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:30.454663 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:30.455142 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:30.455272 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:30.456901 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:30.456902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:30.456903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:30.456910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:30.458428 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:30.458860 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:30.458861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:30.458862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:30.458925 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:30.459159 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:30.459160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:30.459160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:30.459281 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:30.459418 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:30.459551 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 08:41:30.459633 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:41:30.460659 8999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9e2fe2da2d2495fada2e33d4d19e9ad: Generated new TSK 0
I20260807 08:41:30.460879 8999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:30.465878 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 8867
W20260807 08:41:30.480571 8847 connection.cc:535] client connection to 127.0.0.1:42933 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42933): Connection reset by peer (error 104)
2026-08-07T08:41:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:30 dist-test-slave-h6n0 krb5kdc[9027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:30 dist-test-slave-h6n0 krb5kdc[9027](info): set up 2 sockets
Aug 07 08:41:30 dist-test-slave-h6n0 krb5kdc[9027](info): commencing operation
krb5kdc: starting...
W20260807 08:41:31.742969 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:31 dist-test-slave-h6n0 krb5kdc[9027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:31.872771 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:31.955816 9047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:31.956007 9047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:31.956028 9047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:31.957130 9047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:31.957175 9047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:31.957190 9047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:31.957204 9047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:31.957216 9047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:31.958875 9047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:31.959131 9047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:31.959364 9047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:31.964707 9059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:31.964676 9054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:31.964874 9047 server_base.cc:1034] running on GCE node
W20260807 08:41:31.964775 9056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:31.965327 9047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:31.965687 9047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:31.966835 9047 hybrid_clock.cc:648] HybridClock initialized: now 1786092091966821 us; error 34 us; skew 500 ppm
Aug 07 08:41:31 dist-test-slave-h6n0 krb5kdc[9027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092091, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:31.969944 9047 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:31.970515 9047 webserver.cc:460] Webserver started at http://127.0.0.1:44267/ using document root <none> and password file <none>
I20260807 08:41:31.970882 9047 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:31.970939 9047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:31.971086 9047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:31.972332 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "4b6c6386f57247b1bd264510b3e21ada"
format_stamp: "Formatted at 2026-08-07 08:41:31 on dist-test-slave-h6n0"
server_key: "d45846403e3b7d286f4af078a842e099"
server_key_iv: "97192cad34de4be8f8554ea6df46387f"
server_key_version: "encryptionkey@0"
I20260807 08:41:31.972733 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "4b6c6386f57247b1bd264510b3e21ada"
format_stamp: "Formatted at 2026-08-07 08:41:31 on dist-test-slave-h6n0"
server_key: "d45846403e3b7d286f4af078a842e099"
server_key_iv: "97192cad34de4be8f8554ea6df46387f"
server_key_version: "encryptionkey@0"
I20260807 08:41:31.974280 9047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:31.975216 9068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:31.975459 9047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:31.975570 9047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "4b6c6386f57247b1bd264510b3e21ada"
format_stamp: "Formatted at 2026-08-07 08:41:31 on dist-test-slave-h6n0"
server_key: "d45846403e3b7d286f4af078a842e099"
server_key_iv: "97192cad34de4be8f8554ea6df46387f"
server_key_version: "encryptionkey@0"
I20260807 08:41:31.975651 9047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:31.997628 9047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:32.000049 9047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:32.000236 9047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:32.005720 9047 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37635
I20260807 08:41:32.005816 9142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37635 every 8 connection(s)
I20260807 08:41:32.006237 9047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:32.007323 9145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:32.008581 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 9047
I20260807 08:41:32.008701 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:32.008955 11068 external_mini_cluster.cc:1428] Setting key fe726c6a141157024560da528268cab3
I20260807 08:41:32.010288 9145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada: Bootstrap starting.
I20260807 08:41:32.011027 9145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:32.011449 9145 log.cc:824] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:32.012228 9145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada: No bootstrap required, opened a new log
I20260807 08:41:32.014791 9145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6c6386f57247b1bd264510b3e21ada" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260807 08:41:32.014956 9145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:32.014997 9145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6c6386f57247b1bd264510b3e21ada, State: Initialized, Role: FOLLOWER
I20260807 08:41:32.015172 9145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6c6386f57247b1bd264510b3e21ada" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260807 08:41:32.015278 9145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:32.015316 9145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:32.015383 9145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:32.016093 9145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6c6386f57247b1bd264510b3e21ada" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260807 08:41:32.016187 9145 leader_election.cc:302] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b6c6386f57247b1bd264510b3e21ada; no voters:
I20260807 08:41:32.016191 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:32.009614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56449 (local address 127.0.0.1:37635)
0807 08:41:32.009850 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:32.009861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:32.010170 (+ 309us) server_negotiation.cc:407] Connection header received
0807 08:41:32.010947 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:32.010957 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:32.011021 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:32.011263 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:32.011561 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:32.012678 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:32.013803 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:32.014288 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:32.014391 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:32.015347 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:32.015352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:32.015352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:32.015592 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:32.015630 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:32.015642 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:32.015718 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:32.015866 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 08:41:32.015915 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260807 08:41:32.016445 9145 leader_election.cc:288] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:32.016510 9154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:32.016705 9154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [term 1 LEADER]: Becoming Leader. State: Replica: 4b6c6386f57247b1bd264510b3e21ada, State: Running, Role: LEADER
I20260807 08:41:32.016822 9154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6c6386f57247b1bd264510b3e21ada" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260807 08:41:32.016939 9145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:32.017263 9154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b6c6386f57247b1bd264510b3e21ada. Latest consensus state: current_term: 1 leader_uuid: "4b6c6386f57247b1bd264510b3e21ada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6c6386f57247b1bd264510b3e21ada" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260807 08:41:32.017290 9156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b6c6386f57247b1bd264510b3e21ada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6c6386f57247b1bd264510b3e21ada" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260807 08:41:32.017356 9154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:32.017406 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:32.017421 9156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:32.017607 9162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:32.018082 9162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:32.020318 9162 catalog_manager.cc:1347] Generated new cluster ID: 77bf7ae98e5f4586ada3c7692728866d
I20260807 08:41:32.020383 9162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:32.023518 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:32.019172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57449 (local address 127.0.0.1:37635)
0807 08:41:32.019211 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:32.019213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:32.019807 (+ 594us) server_negotiation.cc:407] Connection header received
0807 08:41:32.019911 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:32.019912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:32.019923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:32.019955 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:32.020201 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:32.020969 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:32.022015 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:32.022363 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:32.022452 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:32.023212 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:32.023214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:32.023214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:32.023219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:32.023238 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:32.023243 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:32.023321 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:32.023425 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 08:41:32.023438 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:41:32.031881 9162 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:32.032665 9162 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:32.035382 9162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b6c6386f57247b1bd264510b3e21ada: Generated new TSK 0
I20260807 08:41:32.035569 9162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:32.038825 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 9047
W20260807 08:41:32.062508 9024 connection.cc:535] client connection to 127.0.0.1:37635 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37635): Connection reset by peer (error 104)
2026-08-07T08:41:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1581 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:32 dist-test-slave-h6n0 krb5kdc[9197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:32 dist-test-slave-h6n0 krb5kdc[9197](info): set up 2 sockets
Aug 07 08:41:32 dist-test-slave-h6n0 krb5kdc[9197](info): commencing operation
krb5kdc: starting...
W20260807 08:41:33.331696 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:33 dist-test-slave-h6n0 krb5kdc[9197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:33.463284 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:33.549355 9216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:33.549564 9216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:33.549588 9216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:33.550720 9216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:33.550765 9216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:33.550779 9216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:33.550792 9216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:33.550805 9216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:33.552281 9216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44473
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:33.552547 9216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:33.552778 9216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:33.558149 9224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:33.558199 9221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:33.558259 9222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:33.558518 9216 server_base.cc:1034] running on GCE node
I20260807 08:41:33.558830 9216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:33.559202 9216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:33.560588 9216 hybrid_clock.cc:648] HybridClock initialized: now 1786092093560567 us; error 55 us; skew 500 ppm
Aug 07 08:41:33 dist-test-slave-h6n0 krb5kdc[9197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092093, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:33.563827 9216 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:33.564379 9216 webserver.cc:460] Webserver started at http://127.0.0.1:41701/ using document root <none> and password file <none>
I20260807 08:41:33.564642 9216 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:33.564715 9216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:33.564874 9216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:33.565922 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "c5bf4b69b2004033a40155dbe8bba623"
format_stamp: "Formatted at 2026-08-07 08:41:33 on dist-test-slave-h6n0"
server_key: "69d230ae2c94435920ad6e4aed7522ea"
server_key_iv: "4ce5de0e1322c561babde4225614ff83"
server_key_version: "encryptionkey@0"
I20260807 08:41:33.566301 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "c5bf4b69b2004033a40155dbe8bba623"
format_stamp: "Formatted at 2026-08-07 08:41:33 on dist-test-slave-h6n0"
server_key: "69d230ae2c94435920ad6e4aed7522ea"
server_key_iv: "4ce5de0e1322c561babde4225614ff83"
server_key_version: "encryptionkey@0"
I20260807 08:41:33.567818 9216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:33.568722 9232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:33.569056 9216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:33.569180 9216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "c5bf4b69b2004033a40155dbe8bba623"
format_stamp: "Formatted at 2026-08-07 08:41:33 on dist-test-slave-h6n0"
server_key: "69d230ae2c94435920ad6e4aed7522ea"
server_key_iv: "4ce5de0e1322c561babde4225614ff83"
server_key_version: "encryptionkey@0"
I20260807 08:41:33.569259 9216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:33.572726 9216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:33.574937 9216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:33.575098 9216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:33.580276 9216 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44473
I20260807 08:41:33.580336 9309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44473 every 8 connection(s)
I20260807 08:41:33.580682 9216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:33.581717 9310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:33.584415 9310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623: Bootstrap starting.
I20260807 08:41:33.585047 9310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:33.585345 9310 log.cc:824] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:33.586136 9310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623: No bootstrap required, opened a new log
I20260807 08:41:33.587960 9310 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bf4b69b2004033a40155dbe8bba623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44473 } }
I20260807 08:41:33.588138 9310 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:33.588164 9310 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bf4b69b2004033a40155dbe8bba623, State: Initialized, Role: FOLLOWER
I20260807 08:41:33.588280 9310 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bf4b69b2004033a40155dbe8bba623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44473 } }
I20260807 08:41:33.588340 9310 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:33.588361 9310 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:33.588420 9310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:33.589005 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 9216
I20260807 08:41:33.589143 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:33.589152 9310 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bf4b69b2004033a40155dbe8bba623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44473 } }
I20260807 08:41:33.589267 9310 leader_election.cc:302] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5bf4b69b2004033a40155dbe8bba623; no voters:
I20260807 08:41:33.589396 11068 external_mini_cluster.cc:1428] Setting key 43f81a8406be69730a874460c75f08c0
I20260807 08:41:33.589471 9310 leader_election.cc:288] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:33.589553 9316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:33.589879 9310 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:33.589859 9316 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [term 1 LEADER]: Becoming Leader. State: Replica: c5bf4b69b2004033a40155dbe8bba623, State: Running, Role: LEADER
I20260807 08:41:33.590076 9316 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bf4b69b2004033a40155dbe8bba623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44473 } }
I20260807 08:41:33.590677 9317 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5bf4b69b2004033a40155dbe8bba623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bf4b69b2004033a40155dbe8bba623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44473 } } }
I20260807 08:41:33.590979 9317 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:33.590875 9318 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5bf4b69b2004033a40155dbe8bba623. Latest consensus state: current_term: 1 leader_uuid: "c5bf4b69b2004033a40155dbe8bba623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bf4b69b2004033a40155dbe8bba623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44473 } } }
I20260807 08:41:33.591346 9318 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623 [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:33.592468 9345 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:33.592538 9345 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:33.592698 9333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:33.593040 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:33.590164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54389 (local address 127.0.0.1:44473)
0807 08:41:33.590381 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:33.590391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:33.591163 (+ 772us) server_negotiation.cc:407] Connection header received
0807 08:41:33.591324 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:33.591333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:33.591370 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:33.591518 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:33.592595 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:33.592599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:33.592599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:33.592604 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:33.592648 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:33.592664 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:33.592747 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:33.592836 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 08:41:33.592883 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260807 08:41:33.593327 9333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:33.594316 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:33.597380 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:33.596062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54919 (local address 127.0.0.1:44473)
0807 08:41:33.596144 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:33.596146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:33.596286 (+ 140us) server_negotiation.cc:407] Connection header received
0807 08:41:33.596372 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:33.596373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:33.596380 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:33.596430 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:33.597089 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:33.597091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:33.597125 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:33.597130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:33.597152 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:33.597157 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:33.597215 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:33.597276 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 08:41:33.597294 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:41:33.599022 9333 catalog_manager.cc:1347] Generated new cluster ID: 7404fac44ec3493ebe80386513e331d0
I20260807 08:41:33.599086 9333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:33.605475 9333 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:33.605684 9333 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:33.612149 9333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5bf4b69b2004033a40155dbe8bba623: Generated new TSK 0
I20260807 08:41:33.612350 9333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:33.617290 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 9216
2026-08-07T08:41:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:33 dist-test-slave-h6n0 krb5kdc[9360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:33 dist-test-slave-h6n0 krb5kdc[9360](info): set up 2 sockets
Aug 07 08:41:33 dist-test-slave-h6n0 krb5kdc[9360](info): commencing operation
krb5kdc: starting...
W20260807 08:41:34.891500 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:34 dist-test-slave-h6n0 krb5kdc[9360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:35.025074 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:35.108098 9377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:35.108325 9377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:35.108363 9377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:35.109454 9377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:35.109524 9377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:35.109550 9377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:35.109578 9377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:35.109608 9377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:35.111069 9377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:35.111334 9377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:35.111559 9377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:35.116542 9382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:35.116554 9389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:35.116696 9383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:35.116840 9377 server_base.cc:1034] running on GCE node
I20260807 08:41:35.117118 9377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:35.117410 9377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:35.118654 9377 hybrid_clock.cc:648] HybridClock initialized: now 1786092095118632 us; error 80 us; skew 500 ppm
Aug 07 08:41:35 dist-test-slave-h6n0 krb5kdc[9360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092095, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:35.122027 9377 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:35.122722 9377 webserver.cc:460] Webserver started at http://127.0.0.1:46259/ using document root <none> and password file <none>
I20260807 08:41:35.122990 9377 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:35.123071 9377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:35.123224 9377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:35.124574 9377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986"
format_stamp: "Formatted at 2026-08-07 08:41:35 on dist-test-slave-h6n0"
server_key: "e4ef2fcb894f28f44a5cb0641279f517"
server_key_iv: "4ba9ec7953a62886e79716d329f12515"
server_key_version: "encryptionkey@0"
I20260807 08:41:35.125022 9377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986"
format_stamp: "Formatted at 2026-08-07 08:41:35 on dist-test-slave-h6n0"
server_key: "e4ef2fcb894f28f44a5cb0641279f517"
server_key_iv: "4ba9ec7953a62886e79716d329f12515"
server_key_version: "encryptionkey@0"
I20260807 08:41:35.126470 9377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:35.127323 9399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:35.127549 9377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:35.127661 9377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986"
format_stamp: "Formatted at 2026-08-07 08:41:35 on dist-test-slave-h6n0"
server_key: "e4ef2fcb894f28f44a5cb0641279f517"
server_key_iv: "4ba9ec7953a62886e79716d329f12515"
server_key_version: "encryptionkey@0"
I20260807 08:41:35.127746 9377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:35.142788 9377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:35.144852 9377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:35.145012 9377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:35.149892 9377 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33521
I20260807 08:41:35.149948 9482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33521 every 8 connection(s)
I20260807 08:41:35.150301 9377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:35.151418 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 9377
I20260807 08:41:35.151531 9484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:35.151579 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:35.151809 11068 external_mini_cluster.cc:1428] Setting key cec505e1a36502de60769a4e3853df3d
I20260807 08:41:35.155155 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986: Bootstrap starting.
I20260807 08:41:35.155869 9484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:35.156206 9484 log.cc:824] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:35.156948 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986: No bootstrap required, opened a new log
Aug 07 08:41:35 dist-test-slave-h6n0 krb5kdc[9360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:41:35.159370 9484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260807 08:41:35.159543 9484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:35.159595 9484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad3ebafb2de4bebb1fa9b6e5f974986, State: Initialized, Role: FOLLOWER
I20260807 08:41:35.159780 9484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260807 08:41:35.159881 9484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:35.159927 9484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:35.159976 9484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:35.160610 9484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260807 08:41:35.160771 9484 leader_election.cc:302] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ad3ebafb2de4bebb1fa9b6e5f974986; no voters:
I20260807 08:41:35.160988 9484 leader_election.cc:288] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:35.161023 9490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:35.161247 9490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [term 1 LEADER]: Becoming Leader. State: Replica: 1ad3ebafb2de4bebb1fa9b6e5f974986, State: Running, Role: LEADER
I20260807 08:41:35.161309 9484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:35.161387 9490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260807 08:41:35.161953 9492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } } }
I20260807 08:41:35.162055 9492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:35.162321 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ad3ebafb2de4bebb1fa9b6e5f974986. Latest consensus state: current_term: 1 leader_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ad3ebafb2de4bebb1fa9b6e5f974986" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } } }
I20260807 08:41:35.162729 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:35.162914 9501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:35.163483 9501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:35.164104 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:35.152386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33547 (local address 127.0.0.1:33521)
0807 08:41:35.152637 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:35.152645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:35.152669 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:41:35.153226 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:35.153231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:35.153275 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:35.153423 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:35.154075 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:35.155065 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:35.155864 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:35.156373 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:35.156491 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:35.159481 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:35.159490 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:35.159490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:35.159504 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:35.162011 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:35.162547 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:35.162551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:35.162551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:35.162724 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:35.163038 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:35.163039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:35.163039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:35.163397 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:35.163625 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:35.163733 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 08:41:35.163889 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":187,"thread_start_us":105,"threads_started":1}
I20260807 08:41:35.165259 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:35.167042 9501 catalog_manager.cc:1347] Generated new cluster ID: 613cb4fcd6e04929b7ef62afc1f2a25f
I20260807 08:41:35.167115 9501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:35.175381 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:35.167149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47865 (local address 127.0.0.1:33521)
0807 08:41:35.167188 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:35.167190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:35.167363 (+ 173us) server_negotiation.cc:407] Connection header received
0807 08:41:35.167464 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:35.167465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:35.167476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:35.167511 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:35.168441 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:35.169234 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:35.170360 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:35.170736 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:35.170783 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:35.172381 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:35.172382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:35.172383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:35.172390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:35.174016 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:35.174422 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:35.174423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:35.174424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:35.174480 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:35.174751 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:35.174752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:35.174753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:35.174887 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:35.174965 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:35.175188 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 08:41:35.175270 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:41:35.193395 9501 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:35.194077 9501 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:35.201395 9501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ad3ebafb2de4bebb1fa9b6e5f974986: Generated new TSK 0
I20260807 08:41:35.201591 9501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:35.204288 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 9377
2026-08-07T08:41:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:35 dist-test-slave-h6n0 krb5kdc[9526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:35 dist-test-slave-h6n0 krb5kdc[9526](info): set up 2 sockets
Aug 07 08:41:35 dist-test-slave-h6n0 krb5kdc[9526](info): commencing operation
krb5kdc: starting...
W20260807 08:41:36.465569 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:36 dist-test-slave-h6n0 krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:36.607774 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:36.704445 9546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:36.704710 9546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:36.704738 9546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:36.706015 9546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:36.706068 9546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:36.706084 9546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:36.706097 9546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:36.706110 9546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:36.707762 9546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:36.708101 9546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:36.708384 9546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:36.713937 9558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:36.714053 9554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:36.713948 9556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:36.714221 9546 server_base.cc:1034] running on GCE node
I20260807 08:41:36.714673 9546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:36.714989 9546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:36.716162 9546 hybrid_clock.cc:648] HybridClock initialized: now 1786092096716143 us; error 42 us; skew 500 ppm
Aug 07 08:41:36 dist-test-slave-h6n0 krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:36.722949 9546 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:36.723542 9546 webserver.cc:460] Webserver started at http://127.0.0.1:40457/ using document root <none> and password file <none>
I20260807 08:41:36.723807 9546 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:36.723879 9546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:36.724040 9546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:36.725644 9546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "fed3da7cde5846af931b1691d96ca411"
format_stamp: "Formatted at 2026-08-07 08:41:36 on dist-test-slave-h6n0"
server_key: "efce42ad798dd281c8ff7acff067c2c0"
server_key_iv: "bbf77731dcd8e259f839354175d737cf"
server_key_version: "encryptionkey@0"
I20260807 08:41:36.726212 9546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "fed3da7cde5846af931b1691d96ca411"
format_stamp: "Formatted at 2026-08-07 08:41:36 on dist-test-slave-h6n0"
server_key: "efce42ad798dd281c8ff7acff067c2c0"
server_key_iv: "bbf77731dcd8e259f839354175d737cf"
server_key_version: "encryptionkey@0"
I20260807 08:41:36.727896 9546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 08:41:36.728966 9573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:36.729250 9546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:36.729377 9546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "fed3da7cde5846af931b1691d96ca411"
format_stamp: "Formatted at 2026-08-07 08:41:36 on dist-test-slave-h6n0"
server_key: "efce42ad798dd281c8ff7acff067c2c0"
server_key_iv: "bbf77731dcd8e259f839354175d737cf"
server_key_version: "encryptionkey@0"
I20260807 08:41:36.729490 9546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:36.737363 9546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:36.739856 9546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:36.740049 9546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:36.745543 9546 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32861
I20260807 08:41:36.745599 9647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32861 every 8 connection(s)
I20260807 08:41:36.746028 9546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:36.747251 9648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:36.749878 9648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411: Bootstrap starting.
I20260807 08:41:36.750437 9648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:36.750770 9648 log.cc:824] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:36.751654 9648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411: No bootstrap required, opened a new log
I20260807 08:41:36.753280 9648 raft_consensus.cc:348] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed3da7cde5846af931b1691d96ca411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32861 } }
I20260807 08:41:36.753435 9648 raft_consensus.cc:374] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:36.753466 9648 raft_consensus.cc:729] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed3da7cde5846af931b1691d96ca411, State: Initialized, Role: FOLLOWER
I20260807 08:41:36.753572 9648 consensus_queue.cc:260] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed3da7cde5846af931b1691d96ca411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32861 } }
I20260807 08:41:36.753631 9648 raft_consensus.cc:388] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:36.753654 9648 raft_consensus.cc:482] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:36.753718 9648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:36.754292 9648 raft_consensus.cc:504] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed3da7cde5846af931b1691d96ca411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32861 } }
I20260807 08:41:36.754386 9648 leader_election.cc:302] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fed3da7cde5846af931b1691d96ca411; no voters:
I20260807 08:41:36.754642 9648 leader_election.cc:288] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:36.754724 9652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:36.754938 9648 sys_catalog.cc:564] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:36.755015 9652 raft_consensus.cc:686] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [term 1 LEADER]: Becoming Leader. State: Replica: fed3da7cde5846af931b1691d96ca411, State: Running, Role: LEADER
I20260807 08:41:36.755131 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 9546
I20260807 08:41:36.755276 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:36.755216 9652 consensus_queue.cc:237] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed3da7cde5846af931b1691d96ca411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32861 } }
I20260807 08:41:36.755484 11068 external_mini_cluster.cc:1428] Setting key c5e4688753a7f8abe2d550e5da4de8ea
I20260807 08:41:36.755906 9653 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fed3da7cde5846af931b1691d96ca411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed3da7cde5846af931b1691d96ca411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32861 } } }
I20260807 08:41:36.756013 9653 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:36.756685 9668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:36.757124 9656 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fed3da7cde5846af931b1691d96ca411. Latest consensus state: current_term: 1 leader_uuid: "fed3da7cde5846af931b1691d96ca411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed3da7cde5846af931b1691d96ca411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32861 } } }
I20260807 08:41:36.757272 9656 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:36.757548 9668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:36.759393 9668 catalog_manager.cc:1347] Generated new cluster ID: 236c0c8af0144e028f61c2abf6f11d08
I20260807 08:41:36.759455 9668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:36.763142 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:36.756124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34529 (local address 127.0.0.1:32861)
0807 08:41:36.756400 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:36.756410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:36.756444 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:41:36.756571 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:36.756578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:36.756929 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:36.758331 (+ 1402us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:36.758731 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:36.760055 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:36.760901 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:36.761431 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:36.761565 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:36.762348 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:36.762352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:36.762352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:36.762632 (+ 280us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:36.762681 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:36.762699 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:36.762769 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:36.762849 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 08:41:36.762921 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
I20260807 08:41:36.764158 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:36.769842 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:36.765691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59247 (local address 127.0.0.1:32861)
0807 08:41:36.765734 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:36.765736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:36.766765 (+ 1029us) server_negotiation.cc:407] Connection header received
0807 08:41:36.766844 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:36.766846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:36.766862 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:36.766923 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:36.767119 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:36.767742 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:36.768512 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:36.768735 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:36.768812 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:36.769559 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:36.769560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:36.769561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:36.769568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:36.769583 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:36.769588 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:36.769642 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:36.769707 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:41:36.769733 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:41:36.777097 9668 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:36.777662 9668 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:36.783897 9668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fed3da7cde5846af931b1691d96ca411: Generated new TSK 0
I20260807 08:41:36.784155 9668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:36.792717 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 9546
2026-08-07T08:41:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:36 dist-test-slave-h6n0 krb5kdc[9696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:36 dist-test-slave-h6n0 krb5kdc[9696](info): set up 2 sockets
Aug 07 08:41:36 dist-test-slave-h6n0 krb5kdc[9696](info): commencing operation
krb5kdc: starting...
W20260807 08:41:38.111861 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:38 dist-test-slave-h6n0 krb5kdc[9696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:38.252061 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:38.352643 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:38.352882 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:38.352905 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:38.353963 9716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:38.354005 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:38.354020 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:38.354033 9716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:38.354046 9716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:38.355475 9716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:38.355670 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:38.355836 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:38.360991 9729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:38.361004 9724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:38.361294 9716 server_base.cc:1034] running on GCE node
W20260807 08:41:38.360998 9727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:38.361819 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:38.362169 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:38.363333 9716 hybrid_clock.cc:648] HybridClock initialized: now 1786092098363321 us; error 33 us; skew 500 ppm
Aug 07 08:41:38 dist-test-slave-h6n0 krb5kdc[9696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:38.366356 9716 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:38.366973 9716 webserver.cc:460] Webserver started at http://127.0.0.1:33457/ using document root <none> and password file <none>
I20260807 08:41:38.367205 9716 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:38.367254 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:38.367363 9716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:38.368614 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "d3431a177352436384d1a56c0bf201c3"
format_stamp: "Formatted at 2026-08-07 08:41:38 on dist-test-slave-h6n0"
server_key: "d78364413357ecbb65e187d5f46a568d"
server_key_iv: "3b13e5d94e8c291df2d2c18ca1fd3f20"
server_key_version: "encryptionkey@0"
I20260807 08:41:38.369012 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "d3431a177352436384d1a56c0bf201c3"
format_stamp: "Formatted at 2026-08-07 08:41:38 on dist-test-slave-h6n0"
server_key: "d78364413357ecbb65e187d5f46a568d"
server_key_iv: "3b13e5d94e8c291df2d2c18ca1fd3f20"
server_key_version: "encryptionkey@0"
I20260807 08:41:38.370596 9716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:38.371560 9739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:38.371773 9716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:38.371886 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "d3431a177352436384d1a56c0bf201c3"
format_stamp: "Formatted at 2026-08-07 08:41:38 on dist-test-slave-h6n0"
server_key: "d78364413357ecbb65e187d5f46a568d"
server_key_iv: "3b13e5d94e8c291df2d2c18ca1fd3f20"
server_key_version: "encryptionkey@0"
I20260807 08:41:38.371969 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:38.375347 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:38.377600 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:38.377758 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:38.382064 9716 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44201
I20260807 08:41:38.382169 9817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44201 every 8 connection(s)
I20260807 08:41:38.382496 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:38.383559 9818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:38.386129 9818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3: Bootstrap starting.
I20260807 08:41:38.386865 9818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:38.387142 9818 log.cc:824] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:38.387858 9818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3: No bootstrap required, opened a new log
I20260807 08:41:38.388602 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 9716
I20260807 08:41:38.388727 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:38.388942 11068 external_mini_cluster.cc:1428] Setting key fda94e6b197dc6914fcbadffde407ca7
I20260807 08:41:38.390491 9818 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3431a177352436384d1a56c0bf201c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260807 08:41:38.390753 9818 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:38.390811 9818 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3431a177352436384d1a56c0bf201c3, State: Initialized, Role: FOLLOWER
I20260807 08:41:38.391026 9818 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3431a177352436384d1a56c0bf201c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260807 08:41:38.391137 9818 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:38.391182 9818 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:38.391249 9818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:38.392150 9818 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3431a177352436384d1a56c0bf201c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260807 08:41:38.392258 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:38.389532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43729 (local address 127.0.0.1:44201)
0807 08:41:38.389724 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:38.389732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:38.389758 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:41:38.390686 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:38.390693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:38.390721 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:38.390880 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:38.391717 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:38.391722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:38.391722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:38.391728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:38.391784 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:38.391800 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:38.391894 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:38.391971 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 08:41:38.392020 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260807 08:41:38.392273 9818 leader_election.cc:302] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3431a177352436384d1a56c0bf201c3; no voters:
I20260807 08:41:38.392477 9818 leader_election.cc:288] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:38.392508 9826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:38.392791 9818 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:38.392894 9826 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [term 1 LEADER]: Becoming Leader. State: Replica: d3431a177352436384d1a56c0bf201c3, State: Running, Role: LEADER
I20260807 08:41:38.393106 9826 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3431a177352436384d1a56c0bf201c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } }
I20260807 08:41:38.393499 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 08:41:38.395190 9845 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:38.395327 9845 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:38.395321 9828 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3431a177352436384d1a56c0bf201c3. Latest consensus state: current_term: 1 leader_uuid: "d3431a177352436384d1a56c0bf201c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3431a177352436384d1a56c0bf201c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } } }
I20260807 08:41:38.395527 9827 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3431a177352436384d1a56c0bf201c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3431a177352436384d1a56c0bf201c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44201 } } }
I20260807 08:41:38.395573 9828 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:38.395593 9827 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:38.395920 9853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:38.396159 9853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:38.396690 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:38.395147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41959 (local address 127.0.0.1:44201)
0807 08:41:38.395193 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:38.395196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:38.395427 (+ 231us) server_negotiation.cc:407] Connection header received
0807 08:41:38.395524 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:38.395526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:38.395531 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:38.395562 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:38.396442 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:38.396443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:38.396444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:38.396449 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:38.396463 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:38.396469 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:38.396499 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:38.396609 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 08:41:38.396623 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:41:38.399811 9853 catalog_manager.cc:1347] Generated new cluster ID: 14f3240c9d80445085c7bcb61630b8b0
I20260807 08:41:38.399942 9853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:38.441299 9853 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:38.441587 9853 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:38.449008 9853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3431a177352436384d1a56c0bf201c3: Generated new TSK 0
I20260807 08:41:38.449246 9853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:38.460969 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 9716
2026-08-07T08:41:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:38 dist-test-slave-h6n0 krb5kdc[9857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:38 dist-test-slave-h6n0 krb5kdc[9857](info): set up 2 sockets
Aug 07 08:41:38 dist-test-slave-h6n0 krb5kdc[9857](info): commencing operation
krb5kdc: starting...
W20260807 08:41:39.734277 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:39 dist-test-slave-h6n0 krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:39.869447 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:39.961275 9880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:39.961562 9880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:39.961601 9880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:39.962796 9880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:39.962870 9880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:39.962898 9880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:39.962927 9880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:39.962960 9880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:39.964542 9880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:39.964854 9880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:39.965118 9880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:39.971067 9887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:39.971086 9890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:39.971086 9888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:39.971477 9880 server_base.cc:1034] running on GCE node
I20260807 08:41:39.971743 9880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:39.972041 9880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:39.973232 9880 hybrid_clock.cc:648] HybridClock initialized: now 1786092099973201 us; error 54 us; skew 500 ppm
Aug 07 08:41:39 dist-test-slave-h6n0 krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092099, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:39.976202 9880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:39.976724 9880 webserver.cc:460] Webserver started at http://127.0.0.1:41237/ using document root <none> and password file <none>
I20260807 08:41:39.976980 9880 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:39.977054 9880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:39.977216 9880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:39.978410 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "4da9b4189ba74ac983817d63dfdfae98"
format_stamp: "Formatted at 2026-08-07 08:41:39 on dist-test-slave-h6n0"
server_key: "9518dbfaf3a7dd2c57af699e735e0a34"
server_key_iv: "a8d208979a98a6b89735f80c4d8f547f"
server_key_version: "encryptionkey@0"
I20260807 08:41:39.978899 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "4da9b4189ba74ac983817d63dfdfae98"
format_stamp: "Formatted at 2026-08-07 08:41:39 on dist-test-slave-h6n0"
server_key: "9518dbfaf3a7dd2c57af699e735e0a34"
server_key_iv: "a8d208979a98a6b89735f80c4d8f547f"
server_key_version: "encryptionkey@0"
I20260807 08:41:39.980512 9880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:39.981467 9902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:39.981804 9880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:39.981956 9880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "4da9b4189ba74ac983817d63dfdfae98"
format_stamp: "Formatted at 2026-08-07 08:41:39 on dist-test-slave-h6n0"
server_key: "9518dbfaf3a7dd2c57af699e735e0a34"
server_key_iv: "a8d208979a98a6b89735f80c4d8f547f"
server_key_version: "encryptionkey@0"
I20260807 08:41:39.982045 9880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:40.008852 9880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:40.011415 9880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:40.011620 9880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:40.016990 9880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40783
I20260807 08:41:40.017055 9989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40783 every 8 connection(s)
I20260807 08:41:40.017503 9880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:40.018576 9991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:40.021083 9991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98: Bootstrap starting.
I20260807 08:41:40.021698 9991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:40.022017 9991 log.cc:824] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:40.022858 9991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98: No bootstrap required, opened a new log
I20260807 08:41:40.024492 9991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da9b4189ba74ac983817d63dfdfae98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260807 08:41:40.024636 9991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:40.024658 9991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da9b4189ba74ac983817d63dfdfae98, State: Initialized, Role: FOLLOWER
I20260807 08:41:40.024813 9991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da9b4189ba74ac983817d63dfdfae98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260807 08:41:40.024915 9991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:40.024937 9991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:40.024966 9991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:40.025545 9991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da9b4189ba74ac983817d63dfdfae98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260807 08:41:40.025683 9991 leader_election.cc:302] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4da9b4189ba74ac983817d63dfdfae98; no voters:
I20260807 08:41:40.025780 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 9880
I20260807 08:41:40.025861 9991 leader_election.cc:288] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:40.025908 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:40.025944 9995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:40.026132 11068 external_mini_cluster.cc:1428] Setting key bf32f1d0d98df7067d8543b45974201e
I20260807 08:41:40.026239 9995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [term 1 LEADER]: Becoming Leader. State: Replica: 4da9b4189ba74ac983817d63dfdfae98, State: Running, Role: LEADER
I20260807 08:41:40.026365 9995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da9b4189ba74ac983817d63dfdfae98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260807 08:41:40.026240 9991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:40.027427 9998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4da9b4189ba74ac983817d63dfdfae98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da9b4189ba74ac983817d63dfdfae98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } } }
I20260807 08:41:40.027545 9998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:40.027825 9999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4da9b4189ba74ac983817d63dfdfae98. Latest consensus state: current_term: 1 leader_uuid: "4da9b4189ba74ac983817d63dfdfae98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da9b4189ba74ac983817d63dfdfae98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } } }
I20260807 08:41:40.027977 9999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:40.028066 10013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:40.028707 10013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:40.030969 10013 catalog_manager.cc:1347] Generated new cluster ID: 02bfa053e52f4008b64384c7c6c565ae
I20260807 08:41:40.031028 10013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:41:40 dist-test-slave-h6n0 krb5kdc[9857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:41:40.039278 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:40.026881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:33139 (local address 127.0.0.1:40783)
0807 08:41:40.027257 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:40.027292 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:40.027399 (+ 107us) server_negotiation.cc:407] Connection header received
0807 08:41:40.027555 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:40.027565 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:40.027640 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:40.028491 (+ 851us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:40.028922 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:40.030387 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:40.031434 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:40.031912 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:40.032037 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:40.034971 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:40.034976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:40.034976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:40.034991 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:40.037355 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:40.037844 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:40.037848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:40.037848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:40.037970 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:40.038226 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:40.038227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:40.038227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:40.038591 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:40.038814 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:40.038907 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 08:41:40.039025 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":125,"threads_started":1}
I20260807 08:41:40.040419 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:40.045385 10013 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:40.046098 10013 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:40.048980 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:40.042046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:35897 (local address 127.0.0.1:40783)
0807 08:41:40.042099 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:40.042101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:40.042237 (+ 136us) server_negotiation.cc:407] Connection header received
0807 08:41:40.042302 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:40.042303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:40.042317 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:40.042361 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:40.042978 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:40.043515 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:40.044288 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:40.044539 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:40.044634 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:40.046058 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:40.046060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:40.046060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:40.046068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:40.047706 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:40.048112 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:40.048114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:40.048114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:40.048176 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:40.048408 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:40.048410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:40.048410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:40.048527 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:40.048658 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:40.048787 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 08:41:40.048854 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:41:40.049743 10013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4da9b4189ba74ac983817d63dfdfae98: Generated new TSK 0
I20260807 08:41:40.049921 10013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:40.055827 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 9880
W20260807 08:41:40.074245 9855 connection.cc:535] client connection to 127.0.0.1:40783 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40783): Connection reset by peer (error 104)
2026-08-07T08:41:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:40 dist-test-slave-h6n0 krb5kdc[10035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:40 dist-test-slave-h6n0 krb5kdc[10035](info): set up 2 sockets
Aug 07 08:41:40 dist-test-slave-h6n0 krb5kdc[10035](info): commencing operation
krb5kdc: starting...
W20260807 08:41:41.341355 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:41 dist-test-slave-h6n0 krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:41.475355 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:41.566866 10050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:41.567173 10050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:41.567214 10050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:41.568434 10050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:41.568504 10050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:41.568529 10050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:41.568552 10050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:41.568583 10050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:41.570178 10050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:41.570479 10050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:41.570763 10050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:41.576342 10058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:41.576377 10060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:41.576366 10057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:41.576705 10050 server_base.cc:1034] running on GCE node
I20260807 08:41:41.577013 10050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:41.577384 10050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:41.578580 10050 hybrid_clock.cc:648] HybridClock initialized: now 1786092101578549 us; error 44 us; skew 500 ppm
Aug 07 08:41:41 dist-test-slave-h6n0 krb5kdc[10035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:41.581429 10050 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:41.581956 10050 webserver.cc:460] Webserver started at http://127.0.0.1:44641/ using document root <none> and password file <none>
I20260807 08:41:41.582216 10050 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:41.582288 10050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:41.582449 10050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:41.583645 10050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527"
format_stamp: "Formatted at 2026-08-07 08:41:41 on dist-test-slave-h6n0"
server_key: "3f2d96fa5b656565c1bb21bbf63c130c"
server_key_iv: "bc8699e2b7a1dae0d8f4e2effd414f10"
server_key_version: "encryptionkey@0"
I20260807 08:41:41.584039 10050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527"
format_stamp: "Formatted at 2026-08-07 08:41:41 on dist-test-slave-h6n0"
server_key: "3f2d96fa5b656565c1bb21bbf63c130c"
server_key_iv: "bc8699e2b7a1dae0d8f4e2effd414f10"
server_key_version: "encryptionkey@0"
I20260807 08:41:41.585505 10050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:41.586364 10070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:41.586624 10050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:41.586736 10050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527"
format_stamp: "Formatted at 2026-08-07 08:41:41 on dist-test-slave-h6n0"
server_key: "3f2d96fa5b656565c1bb21bbf63c130c"
server_key_iv: "bc8699e2b7a1dae0d8f4e2effd414f10"
server_key_version: "encryptionkey@0"
I20260807 08:41:41.586822 10050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:41.609427 10050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:41.611829 10050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:41.612020 10050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:41.617568 10050 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41211
I20260807 08:41:41.617623 10155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41211 every 8 connection(s)
I20260807 08:41:41.618043 10050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:41.619091 10156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:41.621479 10156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527: Bootstrap starting.
I20260807 08:41:41.621518 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 10050
I20260807 08:41:41.621670 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:41.621899 11068 external_mini_cluster.cc:1428] Setting key 1507bcd0714f4f4feb910b91dc163926
I20260807 08:41:41.622237 10156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:41.622853 10156 log.cc:824] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:41.623577 10156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527: No bootstrap required, opened a new log
I20260807 08:41:41.625553 10156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41211 } }
I20260807 08:41:41.625720 10156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:41.625770 10156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bb910d12d3b4a4dbc25b2c86f2ba527, State: Initialized, Role: FOLLOWER
I20260807 08:41:41.625908 10156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41211 } }
I20260807 08:41:41.626009 10156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:41.626061 10156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:41.626111 10156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:41.626789 10156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41211 } }
I20260807 08:41:41.626914 10156 leader_election.cc:302] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bb910d12d3b4a4dbc25b2c86f2ba527; no voters:
I20260807 08:41:41.627108 10156 leader_election.cc:288] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:41.627156 10166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:41.627331 10166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [term 1 LEADER]: Becoming Leader. State: Replica: 5bb910d12d3b4a4dbc25b2c86f2ba527, State: Running, Role: LEADER
I20260807 08:41:41.627391 10156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:41.627480 10166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41211 } }
I20260807 08:41:41.628314 10167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41211 } } }
I20260807 08:41:41.628319 10169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bb910d12d3b4a4dbc25b2c86f2ba527. Latest consensus state: current_term: 1 leader_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb910d12d3b4a4dbc25b2c86f2ba527" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41211 } } }
I20260807 08:41:41.628453 10167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:41.628474 10169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:41.628376 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:41.622447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:44281 (local address 127.0.0.1:41211)
0807 08:41:41.622721 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:41.622730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:41.622759 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:41:41.623376 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:41.623384 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:41.623439 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:41.623604 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:41.623853 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:41.624819 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:41.625635 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:41.626249 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:41.626374 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:41.627153 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:41.627159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:41.627159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:41.627407 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:41.627463 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:41.627481 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:41.627834 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:41.627879 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 08:41:41.627911 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":137,"threads_started":1}
I20260807 08:41:41.629580 10179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:41.629846 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:41.630245 10179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:41.631914 10179 catalog_manager.cc:1347] Generated new cluster ID: d7290d1714a145478c4345641ac15dc7
I20260807 08:41:41.631976 10179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:41.635128 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:41.631608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:47277 (local address 127.0.0.1:41211)
0807 08:41:41.631684 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:41.631687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:41.631844 (+ 157us) server_negotiation.cc:407] Connection header received
0807 08:41:41.631934 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:41.631935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:41.631946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:41.631992 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:41.632197 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:41.632686 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:41.633574 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:41.633942 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:41.633991 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:41.634850 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:41.634852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:41.634852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:41.634857 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:41.634872 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:41.634876 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:41.634935 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:41.635015 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 08:41:41.635029 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:41:41.650300 10179 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:41.651049 10179 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:41.666028 10179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bb910d12d3b4a4dbc25b2c86f2ba527: Generated new TSK 0
I20260807 08:41:41.666221 10179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:41.671125 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 10050
W20260807 08:41:41.685703 10033 connection.cc:535] client connection to 127.0.0.1:41211 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41211): Connection reset by peer (error 104)
2026-08-07T08:41:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:41 dist-test-slave-h6n0 krb5kdc[10203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:41 dist-test-slave-h6n0 krb5kdc[10203](info): set up 2 sockets
Aug 07 08:41:41 dist-test-slave-h6n0 krb5kdc[10203](info): commencing operation
krb5kdc: starting...
W20260807 08:41:42.978448 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:43 dist-test-slave-h6n0 krb5kdc[10203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:43.115737 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:43.203892 10220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:43.204107 10220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:43.204133 10220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:43.205379 10220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:43.205431 10220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:43.205449 10220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:43.205463 10220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:43.205478 10220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:43.207038 10220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:43.207382 10220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:43.207692 10220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:43.212631 10226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:43.212673 10227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:43.212700 10229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:43.212956 10220 server_base.cc:1034] running on GCE node
I20260807 08:41:43.213299 10220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:43.213629 10220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:43.214798 10220 hybrid_clock.cc:648] HybridClock initialized: now 1786092103214766 us; error 49 us; skew 500 ppm
Aug 07 08:41:43 dist-test-slave-h6n0 krb5kdc[10203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092103, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:43.217368 10220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:43.217821 10220 webserver.cc:460] Webserver started at http://127.0.0.1:41097/ using document root <none> and password file <none>
I20260807 08:41:43.218032 10220 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:43.218079 10220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:43.218165 10220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:43.219319 10220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "992972a75c2e41c08157c541bb6c7c76"
format_stamp: "Formatted at 2026-08-07 08:41:43 on dist-test-slave-h6n0"
server_key: "39b7dc58cabfc83ba546f1d07f499656"
server_key_iv: "13177e27b2b00f15425b12d1473e46b3"
server_key_version: "encryptionkey@0"
I20260807 08:41:43.219646 10220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "992972a75c2e41c08157c541bb6c7c76"
format_stamp: "Formatted at 2026-08-07 08:41:43 on dist-test-slave-h6n0"
server_key: "39b7dc58cabfc83ba546f1d07f499656"
server_key_iv: "13177e27b2b00f15425b12d1473e46b3"
server_key_version: "encryptionkey@0"
I20260807 08:41:43.221282 10220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:43.222229 10240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:43.222496 10220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:43.222642 10220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "992972a75c2e41c08157c541bb6c7c76"
format_stamp: "Formatted at 2026-08-07 08:41:43 on dist-test-slave-h6n0"
server_key: "39b7dc58cabfc83ba546f1d07f499656"
server_key_iv: "13177e27b2b00f15425b12d1473e46b3"
server_key_version: "encryptionkey@0"
I20260807 08:41:43.222740 10220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:43.226382 10220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:43.228924 10220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:43.229051 10220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:43.233950 10220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44601
I20260807 08:41:43.234056 10314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44601 every 8 connection(s)
I20260807 08:41:43.234387 10220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:43.235606 10315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:43.237862 10315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76: Bootstrap starting.
I20260807 08:41:43.238427 10315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:43.238881 10315 log.cc:824] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:43.239694 10315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76: No bootstrap required, opened a new log
I20260807 08:41:43.242076 10315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992972a75c2e41c08157c541bb6c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260807 08:41:43.242264 10315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:43.242291 10315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992972a75c2e41c08157c541bb6c7c76, State: Initialized, Role: FOLLOWER
I20260807 08:41:43.242338 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 10220
I20260807 08:41:43.242471 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:43.242489 10315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992972a75c2e41c08157c541bb6c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260807 08:41:43.242727 11068 external_mini_cluster.cc:1428] Setting key 139df672e095e2118f6cdbfa5563bc7c
I20260807 08:41:43.242786 10315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:43.242893 10315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:43.242964 10315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:43.243850 10315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992972a75c2e41c08157c541bb6c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260807 08:41:43.244010 10315 leader_election.cc:302] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 992972a75c2e41c08157c541bb6c7c76; no voters:
I20260807 08:41:43.244220 10315 leader_election.cc:288] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:43.244289 10324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:43.244478 10324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [term 1 LEADER]: Becoming Leader. State: Replica: 992972a75c2e41c08157c541bb6c7c76, State: Running, Role: LEADER
I20260807 08:41:43.244516 10315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:43.244658 10324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992972a75c2e41c08157c541bb6c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260807 08:41:43.246042 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:43.243393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:44815 (local address 127.0.0.1:44601)
0807 08:41:43.244035 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:43.244044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:43.244080 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:41:43.244207 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:43.244215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:43.244246 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:43.244463 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:43.245227 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:43.245232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:43.245233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:43.245239 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:43.245306 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:43.245325 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:43.245413 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:43.245753 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 08:41:43.245809 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":495,"threads_started":1}
I20260807 08:41:43.246241 10325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "992972a75c2e41c08157c541bb6c7c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992972a75c2e41c08157c541bb6c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } } }
I20260807 08:41:43.246337 10325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:43.246851 10326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 992972a75c2e41c08157c541bb6c7c76. Latest consensus state: current_term: 1 leader_uuid: "992972a75c2e41c08157c541bb6c7c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992972a75c2e41c08157c541bb6c7c76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } } }
I20260807 08:41:43.247026 10326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76 [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:43.247228 10341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:43.247311 10341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:43.247462 10342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:43.247459 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:43.247659 10342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:43.249296 10342 catalog_manager.cc:1347] Generated new cluster ID: 2a6f453a720e4ab88bfe45050460394d
I20260807 08:41:43.249352 10342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:43.250576 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:43.249145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:43491 (local address 127.0.0.1:44601)
0807 08:41:43.249236 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:43.249238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:43.249379 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:41:43.249503 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:43.249504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:43.249512 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:43.249581 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:43.250255 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:43.250256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:43.250257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:43.250261 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:43.250278 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:43.250283 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:43.250342 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:43.250449 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 08:41:43.250463 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:41:43.260722 10342 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:43.260941 10342 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:43.267874 10342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 992972a75c2e41c08157c541bb6c7c76: Generated new TSK 0
I20260807 08:41:43.268069 10342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:43.269846 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 10220
2026-08-07T08:41:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:43 dist-test-slave-h6n0 krb5kdc[10355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:43 dist-test-slave-h6n0 krb5kdc[10355](info): set up 2 sockets
Aug 07 08:41:43 dist-test-slave-h6n0 krb5kdc[10355](info): commencing operation
krb5kdc: starting...
W20260807 08:41:44.594836 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:44 dist-test-slave-h6n0 krb5kdc[10355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:44.733269 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:44.819351 10371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:44.819590 10371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:44.819628 10371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:44.820752 10371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:44.820830 10371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:44.820854 10371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:44.820884 10371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:44.820916 10371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:44.822374 10371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41567
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:44.822665 10371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:44.822892 10371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:44.828135 10381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:44.828210 10385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:44.828228 10371 server_base.cc:1034] running on GCE node
W20260807 08:41:44.828130 10380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:44.828831 10371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:44.829202 10371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:44.830358 10371 hybrid_clock.cc:648] HybridClock initialized: now 1786092104830330 us; error 54 us; skew 500 ppm
Aug 07 08:41:44 dist-test-slave-h6n0 krb5kdc[10355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:44.833247 10371 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:44.833767 10371 webserver.cc:460] Webserver started at http://127.0.0.1:45647/ using document root <none> and password file <none>
I20260807 08:41:44.834021 10371 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:44.834093 10371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:44.834260 10371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:44.835528 10371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "dcc19584de2640a9a5aebf173496cb60"
format_stamp: "Formatted at 2026-08-07 08:41:44 on dist-test-slave-h6n0"
server_key: "8ec9347fa02e2760ed29affa9e179f49"
server_key_iv: "dc63e936951e92b0efb4a17999a9e96b"
server_key_version: "encryptionkey@0"
I20260807 08:41:44.835942 10371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "dcc19584de2640a9a5aebf173496cb60"
format_stamp: "Formatted at 2026-08-07 08:41:44 on dist-test-slave-h6n0"
server_key: "8ec9347fa02e2760ed29affa9e179f49"
server_key_iv: "dc63e936951e92b0efb4a17999a9e96b"
server_key_version: "encryptionkey@0"
I20260807 08:41:44.837512 10371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:44.838477 10395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:44.838773 10371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:44.838908 10371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "dcc19584de2640a9a5aebf173496cb60"
format_stamp: "Formatted at 2026-08-07 08:41:44 on dist-test-slave-h6n0"
server_key: "8ec9347fa02e2760ed29affa9e179f49"
server_key_iv: "dc63e936951e92b0efb4a17999a9e96b"
server_key_version: "encryptionkey@0"
I20260807 08:41:44.839001 10371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:44.875329 10371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:44.877629 10371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:44.877825 10371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:44.883327 10371 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41567
I20260807 08:41:44.883385 10472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41567 every 8 connection(s)
I20260807 08:41:44.883886 10371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:44.884953 10473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:44.887389 10473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60: Bootstrap starting.
I20260807 08:41:44.888180 10473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:44.888536 10473 log.cc:824] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:44.889442 10473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60: No bootstrap required, opened a new log
I20260807 08:41:44.890633 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 10371
I20260807 08:41:44.890762 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:44.890974 11068 external_mini_cluster.cc:1428] Setting key a4e31e558a040d4ac70385d0b43db563
I20260807 08:41:44.891615 10473 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19584de2640a9a5aebf173496cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41567 } }
I20260807 08:41:44.891813 10473 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:44.891860 10473 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc19584de2640a9a5aebf173496cb60, State: Initialized, Role: FOLLOWER
I20260807 08:41:44.892019 10473 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19584de2640a9a5aebf173496cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41567 } }
I20260807 08:41:44.892107 10473 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:44.892138 10473 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:44.892192 10473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:44.893303 10473 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19584de2640a9a5aebf173496cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41567 } }
I20260807 08:41:44.893420 10473 leader_election.cc:302] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcc19584de2640a9a5aebf173496cb60; no voters:
I20260807 08:41:44.893601 10473 leader_election.cc:288] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:44.893733 10480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:44.893867 10473 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:44.893996 10480 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [term 1 LEADER]: Becoming Leader. State: Replica: dcc19584de2640a9a5aebf173496cb60, State: Running, Role: LEADER
I20260807 08:41:44.894245 10480 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19584de2640a9a5aebf173496cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41567 } }
I20260807 08:41:44.896040 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcc19584de2640a9a5aebf173496cb60. Latest consensus state: current_term: 1 leader_uuid: "dcc19584de2640a9a5aebf173496cb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19584de2640a9a5aebf173496cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41567 } } }
I20260807 08:41:44.896070 10481 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcc19584de2640a9a5aebf173496cb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19584de2640a9a5aebf173496cb60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41567 } } }
I20260807 08:41:44.896222 10481 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:44.896169 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60 [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:44.897074 10503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:44.897133 10503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:44.897217 10504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:44.897424 10504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:41:44 dist-test-slave-h6n0 krb5kdc[10355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:41:44.899421 10504 catalog_manager.cc:1347] Generated new cluster ID: 86c9d0bf1fc94bf9964b7f31ddf4447f
I20260807 08:41:44.899477 10504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:44.904500 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:44.891659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:46143 (local address 127.0.0.1:41567)
0807 08:41:44.891947 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:44.891956 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:44.891991 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:41:44.892104 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:44.892111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:44.892171 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:44.892355 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:44.893104 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:44.894496 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:44.895951 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:44.896656 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:44.896777 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:44.899905 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:44.899910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:44.899910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:44.899924 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:44.902304 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:44.902791 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:44.902795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:44.902796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:44.902931 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:44.903204 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:44.903205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:44.903206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:44.903582 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:44.903782 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:44.904058 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 08:41:44.904224 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":151,"threads_started":1}
I20260807 08:41:44.905755 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:44.908862 10504 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:44.909484 10504 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:44.911576 10504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcc19584de2640a9a5aebf173496cb60: Generated new TSK 0
I20260807 08:41:44.911787 10504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:44.915266 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:44.907393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:39483 (local address 127.0.0.1:41567)
0807 08:41:44.907505 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:44.907507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:44.907573 (+ 66us) server_negotiation.cc:407] Connection header received
0807 08:41:44.907656 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:44.907657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:44.907669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:44.907727 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:44.908428 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:44.909153 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:44.910290 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:44.910645 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:44.910726 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:44.912179 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:44.912180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:44.912180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:44.912187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:44.913824 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:44.914291 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:44.914293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:44.914293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:44.914365 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:44.914680 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:44.914681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:44.914682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:44.914797 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:44.914916 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:44.915098 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 08:41:44.915166 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 08:41:44.917840 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 10371
W20260807 08:41:44.935321 10353 connection.cc:535] client connection to 127.0.0.1:41567 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41567): Connection reset by peer (error 104)
2026-08-07T08:41:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1654 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:44 dist-test-slave-h6n0 krb5kdc[10518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:44 dist-test-slave-h6n0 krb5kdc[10518](info): set up 2 sockets
Aug 07 08:41:44 dist-test-slave-h6n0 krb5kdc[10518](info): commencing operation
krb5kdc: starting...
W20260807 08:41:46.192116 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:46 dist-test-slave-h6n0 krb5kdc[10518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:46.329449 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:46.422683 10538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:46.422933 10538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:46.422959 10538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:46.424207 10538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:46.424252 10538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:46.424266 10538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:46.424278 10538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:46.424291 10538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:46.425871 10538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:46.426178 10538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:46.426450 10538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:46.432114 10547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:46.432174 10548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:46.432142 10553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:46.432528 10538 server_base.cc:1034] running on GCE node
I20260807 08:41:46.432847 10538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:46.433205 10538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:46.434382 10538 hybrid_clock.cc:648] HybridClock initialized: now 1786092106434359 us; error 36 us; skew 500 ppm
Aug 07 08:41:46 dist-test-slave-h6n0 krb5kdc[10518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:46.437182 10538 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:46.437614 10538 webserver.cc:460] Webserver started at http://127.0.0.1:35409/ using document root <none> and password file <none>
I20260807 08:41:46.437903 10538 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:46.437956 10538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:46.438097 10538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:46.439284 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "1a318ed4b37f4ca487feddc19aebf0c1"
format_stamp: "Formatted at 2026-08-07 08:41:46 on dist-test-slave-h6n0"
server_key: "e972a2404bf9b2ae59ac1af419724de0"
server_key_iv: "7554bd63f414efe7793739d162c4a263"
server_key_version: "encryptionkey@0"
I20260807 08:41:46.439671 10538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "1a318ed4b37f4ca487feddc19aebf0c1"
format_stamp: "Formatted at 2026-08-07 08:41:46 on dist-test-slave-h6n0"
server_key: "e972a2404bf9b2ae59ac1af419724de0"
server_key_iv: "7554bd63f414efe7793739d162c4a263"
server_key_version: "encryptionkey@0"
I20260807 08:41:46.441107 10538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:41:46.441936 10561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:46.442191 10538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:46.442304 10538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "1a318ed4b37f4ca487feddc19aebf0c1"
format_stamp: "Formatted at 2026-08-07 08:41:46 on dist-test-slave-h6n0"
server_key: "e972a2404bf9b2ae59ac1af419724de0"
server_key_iv: "7554bd63f414efe7793739d162c4a263"
server_key_version: "encryptionkey@0"
I20260807 08:41:46.442399 10538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:46.462685 10538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:46.465152 10538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:46.465323 10538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:46.469949 10538 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45675
I20260807 08:41:46.470012 10641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45675 every 8 connection(s)
I20260807 08:41:46.470384 10538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:46.471542 10645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:46.474298 10645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1: Bootstrap starting.
I20260807 08:41:46.475049 10645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:46.475404 10645 log.cc:824] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:46.476269 10645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1: No bootstrap required, opened a new log
I20260807 08:41:46.476325 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 10538
I20260807 08:41:46.476477 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:46.476683 11068 external_mini_cluster.cc:1428] Setting key c358886a61d39884738630de335867ca
I20260807 08:41:46.479907 10645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260807 08:41:46.480091 10645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:46.480125 10645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a318ed4b37f4ca487feddc19aebf0c1, State: Initialized, Role: FOLLOWER
I20260807 08:41:46.480252 10645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260807 08:41:46.480343 10645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:46.480384 10645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:46.480446 10645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:46.481040 10645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260807 08:41:46.481174 10645 leader_election.cc:302] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a318ed4b37f4ca487feddc19aebf0c1; no voters:
I20260807 08:41:46.481395 10645 leader_election.cc:288] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:46.481427 10652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:46.481559 10652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [term 1 LEADER]: Becoming Leader. State: Replica: 1a318ed4b37f4ca487feddc19aebf0c1, State: Running, Role: LEADER
I20260807 08:41:46.481673 10652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } }
I20260807 08:41:46.481716 10645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:46.482724 10654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a318ed4b37f4ca487feddc19aebf0c1. Latest consensus state: current_term: 1 leader_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } } }
I20260807 08:41:46.482689 10653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a318ed4b37f4ca487feddc19aebf0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45675 } } }
I20260807 08:41:46.483032 10654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:46.483055 10653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:46.483434 10669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:46.483976 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:46.477260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:52035 (local address 127.0.0.1:45675)
0807 08:41:46.477466 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:46.477477 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:46.477517 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:41:46.478217 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:46.478227 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:46.478299 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:46.478499 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:46.478779 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:46.480276 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:46.481185 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:46.481932 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:46.482382 (+ 450us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:46.482920 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:46.482924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:46.482925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:46.483197 (+ 272us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:46.483244 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:46.483261 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:46.483666 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:46.483716 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 08:41:46.483746 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260807 08:41:46.484130 10669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:46.485411 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:46.486635 10669 catalog_manager.cc:1347] Generated new cluster ID: d0ed342877a24e6bbce8b1a7f8319afe
I20260807 08:41:46.486732 10669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:46.490906 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:46.487102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:40941 (local address 127.0.0.1:45675)
0807 08:41:46.487147 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:46.487149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:46.487231 (+ 82us) server_negotiation.cc:407] Connection header received
0807 08:41:46.487293 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:46.487294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:46.487306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:46.487383 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:46.487547 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:46.488295 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:46.489357 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:46.489701 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:46.489790 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:46.490569 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:46.490570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:46.490571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:46.490577 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:46.490598 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:46.490604 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:46.490683 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:46.490783 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 08:41:46.490797 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:41:46.510476 10669 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:46.511046 10669 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:46.516796 10669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a318ed4b37f4ca487feddc19aebf0c1: Generated new TSK 0
I20260807 08:41:46.517007 10669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:46.528287 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 10538
W20260807 08:41:46.538229 10516 connection.cc:535] client connection to 127.0.0.1:45675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45675): Connection reset by peer (error 104)
2026-08-07T08:41:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1602 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:46 dist-test-slave-h6n0 krb5kdc[10687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:46 dist-test-slave-h6n0 krb5kdc[10687](info): set up 2 sockets
Aug 07 08:41:46 dist-test-slave-h6n0 krb5kdc[10687](info): commencing operation
krb5kdc: starting...
W20260807 08:41:47.788867 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:47 dist-test-slave-h6n0 krb5kdc[10687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:41:47.924919 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:48.021075 10708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:48.021441 10708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:48.021472 10708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:48.022738 10708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:48.022781 10708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:48.022796 10708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:48.022809 10708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:48.022822 10708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:48.024427 10708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33205
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:48.024700 10708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:48.024928 10708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:48.030607 10718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:48.030786 10708 server_base.cc:1034] running on GCE node
W20260807 08:41:48.030607 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:48.030692 10717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:48.031208 10708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:48.031522 10708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:48.032703 10708 hybrid_clock.cc:648] HybridClock initialized: now 1786092108032678 us; error 43 us; skew 500 ppm
Aug 07 08:41:48 dist-test-slave-h6n0 krb5kdc[10687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:48.040611 10708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:41:48.041096 10708 webserver.cc:460] Webserver started at http://127.0.0.1:45615/ using document root <none> and password file <none>
I20260807 08:41:48.041364 10708 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:48.041438 10708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:48.041605 10708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:48.042804 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "822b01b9b7454db1b2f6398226032930"
format_stamp: "Formatted at 2026-08-07 08:41:48 on dist-test-slave-h6n0"
server_key: "a1144967f13f308174de10c057ab3413"
server_key_iv: "6310ca3ff67d53c56ac0fd18d245d2a9"
server_key_version: "encryptionkey@0"
I20260807 08:41:48.043200 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "822b01b9b7454db1b2f6398226032930"
format_stamp: "Formatted at 2026-08-07 08:41:48 on dist-test-slave-h6n0"
server_key: "a1144967f13f308174de10c057ab3413"
server_key_iv: "6310ca3ff67d53c56ac0fd18d245d2a9"
server_key_version: "encryptionkey@0"
I20260807 08:41:48.044658 10708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:48.045503 10731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:48.045755 10708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:48.045876 10708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "822b01b9b7454db1b2f6398226032930"
format_stamp: "Formatted at 2026-08-07 08:41:48 on dist-test-slave-h6n0"
server_key: "a1144967f13f308174de10c057ab3413"
server_key_iv: "6310ca3ff67d53c56ac0fd18d245d2a9"
server_key_version: "encryptionkey@0"
I20260807 08:41:48.045951 10708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:48.049404 10708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:48.051604 10708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:48.051789 10708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:48.056658 10708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33205
I20260807 08:41:48.056721 10807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33205 every 8 connection(s)
I20260807 08:41:48.057073 10708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:48.058243 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:48.061091 10809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930: Bootstrap starting.
I20260807 08:41:48.061851 10809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:48.062158 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 10708
I20260807 08:41:48.062271 10809 log.cc:824] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:48.062304 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:48.062575 11068 external_mini_cluster.cc:1428] Setting key 8b3e634ddb151aab5ef43aea7d811e39
I20260807 08:41:48.063271 10809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930: No bootstrap required, opened a new log
I20260807 08:41:48.065920 10809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822b01b9b7454db1b2f6398226032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
I20260807 08:41:48.066177 10809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:48.066213 10809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 822b01b9b7454db1b2f6398226032930, State: Initialized, Role: FOLLOWER
I20260807 08:41:48.066365 10809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822b01b9b7454db1b2f6398226032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
I20260807 08:41:48.066443 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:48.063218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:49887 (local address 127.0.0.1:33205)
0807 08:41:48.063497 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:48.063506 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:48.063534 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:41:48.064672 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:48.064680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:48.064709 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:48.064882 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:48.065715 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:48.065719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:48.065720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:48.065724 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:48.065765 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:48.065776 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:48.065852 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:48.065923 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 08:41:48.065970 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":98,"threads_started":1}
I20260807 08:41:48.066514 10809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:48.066623 10809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:48.066684 10809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:48.067672 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:48.067651 10809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822b01b9b7454db1b2f6398226032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
I20260807 08:41:48.067792 10809 leader_election.cc:302] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 822b01b9b7454db1b2f6398226032930; no voters:
I20260807 08:41:48.068012 10809 leader_election.cc:288] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:48.068092 10817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:48.068320 10809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:48.068364 10817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [term 1 LEADER]: Becoming Leader. State: Replica: 822b01b9b7454db1b2f6398226032930, State: Running, Role: LEADER
I20260807 08:41:48.068480 10817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822b01b9b7454db1b2f6398226032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
I20260807 08:41:48.068919 10817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 822b01b9b7454db1b2f6398226032930. Latest consensus state: current_term: 1 leader_uuid: "822b01b9b7454db1b2f6398226032930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822b01b9b7454db1b2f6398226032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } } }
I20260807 08:41:48.068931 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "822b01b9b7454db1b2f6398226032930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822b01b9b7454db1b2f6398226032930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } } }
I20260807 08:41:48.069072 10817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:48.069103 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:48.069996 10843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:48.070638 10843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:48.072216 10857 mvcc.cc:205] Tried to move back new op lower bound from 7315833274661847040 to 7315833274648961024. Current Snapshot: MvccSnapshot[applied={T|T < 7315833274661847040}]
I20260807 08:41:48.072681 10843 catalog_manager.cc:1347] Generated new cluster ID: 78d647fc2e194707882d5ecfcf0815ae
I20260807 08:41:48.072723 10843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:48.075456 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:48.069675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:52049 (local address 127.0.0.1:33205)
0807 08:41:48.070591 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:48.070594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:48.070604 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:41:48.070656 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:48.070657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:48.070664 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:48.070740 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:48.075162 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:48.075165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:48.075166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:48.075172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:48.075197 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:48.075203 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:48.075290 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:48.075368 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 08:41:48.075386 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893}
I20260807 08:41:48.080322 10843 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:48.080607 10843 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:48.086831 10843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 822b01b9b7454db1b2f6398226032930: Generated new TSK 0
I20260807 08:41:48.087038 10843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:48.092110 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 10708
2026-08-07T08:41:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:48 dist-test-slave-h6n0 krb5kdc[10861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:48 dist-test-slave-h6n0 krb5kdc[10861](info): set up 2 sockets
Aug 07 08:41:48 dist-test-slave-h6n0 krb5kdc[10861](info): commencing operation
krb5kdc: starting...
W20260807 08:41:49.369223 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:49 dist-test-slave-h6n0 krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:49.507596 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44387
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:36911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:44387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:49.592662 10877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:49.592900 10877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:49.592941 10877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:49.594046 10877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:49.594126 10877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:49.594154 10877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:49.594182 10877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:49.594211 10877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:49.595746 10877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:44387
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:49.596021 10877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:49.596246 10877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:49.601163 10883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:49.601279 10882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:49.601301 10885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:49.601642 10877 server_base.cc:1034] running on GCE node
I20260807 08:41:49.601960 10877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:49.602367 10877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:49.603549 10877 hybrid_clock.cc:648] HybridClock initialized: now 1786092109603534 us; error 61 us; skew 500 ppm
Aug 07 08:41:49 dist-test-slave-h6n0 krb5kdc[10861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092109, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:49.606792 10877 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:49.607323 10877 webserver.cc:460] Webserver started at http://127.10.207.62:39985/ using document root <none> and password file <none>
I20260807 08:41:49.607618 10877 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:49.607693 10877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:49.607856 10877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:49.609089 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "ba3087d409a4479c8e13c3104bffdc05"
format_stamp: "Formatted at 2026-08-07 08:41:49 on dist-test-slave-h6n0"
server_key: "d55b5d1558a686e23d5531fa94507ea9"
server_key_iv: "57cec537fb5a4b839b7e026ae089691d"
server_key_version: "encryptionkey@0"
I20260807 08:41:49.609493 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "ba3087d409a4479c8e13c3104bffdc05"
format_stamp: "Formatted at 2026-08-07 08:41:49 on dist-test-slave-h6n0"
server_key: "d55b5d1558a686e23d5531fa94507ea9"
server_key_iv: "57cec537fb5a4b839b7e026ae089691d"
server_key_version: "encryptionkey@0"
I20260807 08:41:49.611071 10877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:49.612017 10893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:49.612310 10877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:49.612425 10877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "ba3087d409a4479c8e13c3104bffdc05"
format_stamp: "Formatted at 2026-08-07 08:41:49 on dist-test-slave-h6n0"
server_key: "d55b5d1558a686e23d5531fa94507ea9"
server_key_iv: "57cec537fb5a4b839b7e026ae089691d"
server_key_version: "encryptionkey@0"
I20260807 08:41:49.612519 10877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:49.621694 10877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:49.624240 10877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:49.624415 10877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:49.629889 10877 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:44387
I20260807 08:41:49.630455 10877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:49.630460 10974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:44387 every 8 connection(s)
I20260807 08:41:49.631501 10975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:49.633927 10975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05: Bootstrap starting.
I20260807 08:41:49.634424 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 10877
I20260807 08:41:49.634610 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:49.634642 10975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:49.634876 11068 external_mini_cluster.cc:1428] Setting key ff71773f728cacc8177f1bd0be7a5483
I20260807 08:41:49.634966 10975 log.cc:824] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:49.636034 10975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05: No bootstrap required, opened a new log
I20260807 08:41:49.639122 10975 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3087d409a4479c8e13c3104bffdc05" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44387 } }
I20260807 08:41:49.639338 10975 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:49.639393 10975 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3087d409a4479c8e13c3104bffdc05, State: Initialized, Role: FOLLOWER
I20260807 08:41:49.639561 10975 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3087d409a4479c8e13c3104bffdc05" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44387 } }
I20260807 08:41:49.639672 10975 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:49.639721 10975 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:49.639773 10975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:49.640638 10975 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3087d409a4479c8e13c3104bffdc05" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44387 } }
I20260807 08:41:49.640775 10975 leader_election.cc:302] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba3087d409a4479c8e13c3104bffdc05; no voters:
I20260807 08:41:49.640959 10975 leader_election.cc:288] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:49.641052 10980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:49.641338 10980 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [term 1 LEADER]: Becoming Leader. State: Replica: ba3087d409a4479c8e13c3104bffdc05, State: Running, Role: LEADER
I20260807 08:41:49.641381 10975 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:49.641856 10980 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3087d409a4479c8e13c3104bffdc05" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44387 } }
I20260807 08:41:49.642304 10983 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba3087d409a4479c8e13c3104bffdc05. Latest consensus state: current_term: 1 leader_uuid: "ba3087d409a4479c8e13c3104bffdc05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3087d409a4479c8e13c3104bffdc05" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44387 } } }
I20260807 08:41:49.642395 10983 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:49.642688 10986 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba3087d409a4479c8e13c3104bffdc05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3087d409a4479c8e13c3104bffdc05" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44387 } } }
I20260807 08:41:49.642988 10986 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:49.643173 10994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:49.643719 10994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:41:49 dist-test-slave-h6n0 krb5kdc[10861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:49.645867 10994 catalog_manager.cc:1347] Generated new cluster ID: fbd21497a7fe4246aa6b4de6d2a70add
I20260807 08:41:49.645988 10994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:49.648741 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:49.635561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56767 (local address 127.10.207.62:44387)
0807 08:41:49.635774 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:49.635787 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:49.635830 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:41:49.636587 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:49.636605 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:49.636678 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:49.636862 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:49.637565 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:49.639209 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:49.640224 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:49.640914 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:49.641046 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:49.644640 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:49.644645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:49.644645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:49.644660 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:49.646965 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:49.647411 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:49.647415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:49.647415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:49.647515 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:49.647768 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:49.647769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:49.647769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:49.648050 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:49.648283 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:49.648381 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:41:49.648495 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
I20260807 08:41:49.649988 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:49.654923 10994 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:49.655454 10994 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:49.658804 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:49.651627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51595 (local address 127.10.207.62:44387)
0807 08:41:49.651672 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:49.651674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:49.651811 (+ 137us) server_negotiation.cc:407] Connection header received
0807 08:41:49.651887 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:49.651888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:49.651898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:49.651954 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:49.652542 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:49.653083 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:49.653815 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:49.654088 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:49.654167 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:49.655794 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:49.655796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:49.655796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:49.655802 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:49.657400 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:49.657854 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:49.657855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:49.657855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:49.657916 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:49.658170 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:49.658170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:49.658171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:49.658304 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:49.658436 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:49.658642 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 08:41:49.658712 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:41:49.659799 10994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba3087d409a4479c8e13c3104bffdc05: Generated new TSK 0
I20260807 08:41:49.660004 10994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:49.665853 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 10877
W20260807 08:41:49.680215 10859 connection.cc:535] client connection to 127.10.207.62:44387 recv error: Network error: failed to read from TLS socket (remote: 127.10.207.62:44387): Connection reset by peer (error 104)
2026-08-07T08:41:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:49 dist-test-slave-h6n0 krb5kdc[11018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:49 dist-test-slave-h6n0 krb5kdc[11018](info): set up 2 sockets
Aug 07 08:41:49 dist-test-slave-h6n0 krb5kdc[11018](info): commencing operation
krb5kdc: starting...
W20260807 08:41:50.943089 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:51 dist-test-slave-h6n0 krb5kdc[11018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:51.081153 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:33595
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:33595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:51.173022 11043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:51.173270 11043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:51.173295 11043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:51.174463 11043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:51.174508 11043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:51.174523 11043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:51.174602 11043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:51.174615 11043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:51.176080 11043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:33595
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:33595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:51.176298 11043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:51.176488 11043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:51.181960 11056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:51.181974 11051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:51.181988 11054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:51.182308 11043 server_base.cc:1034] running on GCE node
I20260807 08:41:51.182683 11043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:51.183012 11043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:51.184185 11043 hybrid_clock.cc:648] HybridClock initialized: now 1786092111184161 us; error 39 us; skew 500 ppm
Aug 07 08:41:51 dist-test-slave-h6n0 krb5kdc[11018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:51.186877 11043 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:51.187392 11043 webserver.cc:460] Webserver started at http://127.10.207.62:37611/ using document root <none> and password file <none>
I20260807 08:41:51.187601 11043 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:51.187649 11043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:51.187743 11043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:51.188927 11043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "8ed411b17ffa4330a4562a7c86bffd9b"
format_stamp: "Formatted at 2026-08-07 08:41:51 on dist-test-slave-h6n0"
server_key: "69af023e8e04e1c267ec1bce1ed50503"
server_key_iv: "e4c81abe2f5be2dbae71ca43f79c0168"
server_key_version: "encryptionkey@0"
I20260807 08:41:51.189355 11043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "8ed411b17ffa4330a4562a7c86bffd9b"
format_stamp: "Formatted at 2026-08-07 08:41:51 on dist-test-slave-h6n0"
server_key: "69af023e8e04e1c267ec1bce1ed50503"
server_key_iv: "e4c81abe2f5be2dbae71ca43f79c0168"
server_key_version: "encryptionkey@0"
I20260807 08:41:51.190858 11043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:41:51.191704 11063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:51.191938 11043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:51.192049 11043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "8ed411b17ffa4330a4562a7c86bffd9b"
format_stamp: "Formatted at 2026-08-07 08:41:51 on dist-test-slave-h6n0"
server_key: "69af023e8e04e1c267ec1bce1ed50503"
server_key_iv: "e4c81abe2f5be2dbae71ca43f79c0168"
server_key_version: "encryptionkey@0"
I20260807 08:41:51.192138 11043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:51.211373 11043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:51.213478 11043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:51.213654 11043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:51.218968 11043 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:33595
I20260807 08:41:51.219048 11156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:33595 every 8 connection(s)
I20260807 08:41:51.219431 11043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:51.220435 11160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:51.222875 11160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b: Bootstrap starting.
I20260807 08:41:51.223430 11160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:51.223686 11160 log.cc:824] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:51.224472 11160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b: No bootstrap required, opened a new log
I20260807 08:41:51.226053 11160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33595 } }
I20260807 08:41:51.226193 11160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:51.226215 11160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed411b17ffa4330a4562a7c86bffd9b, State: Initialized, Role: FOLLOWER
I20260807 08:41:51.226321 11160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33595 } }
I20260807 08:41:51.226423 11160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:51.226475 11160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:51.226552 11160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:51.227422 11160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33595 } }
I20260807 08:41:51.227546 11160 leader_election.cc:302] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ed411b17ffa4330a4562a7c86bffd9b; no voters:
I20260807 08:41:51.227732 11160 leader_election.cc:288] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:51.227840 11165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:51.228006 11160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:51.228117 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11043
I20260807 08:41:51.228152 11165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [term 1 LEADER]: Becoming Leader. State: Replica: 8ed411b17ffa4330a4562a7c86bffd9b, State: Running, Role: LEADER
I20260807 08:41:51.228268 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:51.228302 11165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33595 } }
I20260807 08:41:51.228550 11068 external_mini_cluster.cc:1428] Setting key 43852814a42ecbe84dc631e434ff2f29
I20260807 08:41:51.228883 11165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ed411b17ffa4330a4562a7c86bffd9b. Latest consensus state: current_term: 1 leader_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33595 } } }
I20260807 08:41:51.229043 11166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed411b17ffa4330a4562a7c86bffd9b" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 33595 } } }
I20260807 08:41:51.229175 11166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:51.229091 11165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:51.230093 11187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:51.230218 11187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:51.230357 11183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:51.230582 11183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:51.232349 11183 catalog_manager.cc:1347] Generated new cluster ID: d58e3008a45b4e93a0aaf39c6b596aff
I20260807 08:41:51.232404 11183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:51.235327 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:51.229362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48453 (local address 127.10.207.62:33595)
0807 08:41:51.229600 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:51.229611 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:51.229652 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:41:51.229774 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:51.229782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:51.229838 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:51.230309 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:51.230414 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:51.231694 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:51.233008 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:51.233613 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:51.233719 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:51.234576 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:51.234580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:51.234581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:51.234784 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:51.234823 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:51.234837 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:51.234927 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:51.235084 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 08:41:51.235111 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":126,"threads_started":1}
I20260807 08:41:51.236465 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:51.241508 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:51.237884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56489 (local address 127.10.207.62:33595)
0807 08:41:51.237918 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:51.237920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:51.238172 (+ 252us) server_negotiation.cc:407] Connection header received
0807 08:41:51.238252 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:51.238253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:51.238261 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:51.238286 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:51.238471 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:51.239208 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:51.240213 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:51.240532 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:51.240568 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:51.241250 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:51.241251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:51.241252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:51.241256 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:51.241267 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:51.241271 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:51.241299 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:51.241426 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:41:51.241439 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:41:51.243543 11183 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:51.244427 11183 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:51.254252 11183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ed411b17ffa4330a4562a7c86bffd9b: Generated new TSK 0
I20260807 08:41:51.254482 11183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:51.261471 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11043
W20260807 08:41:51.277590 11016 connection.cc:535] client connection to 127.10.207.62:33595 recv error: Network error: failed to read from TLS socket (remote: 127.10.207.62:33595): Connection reset by peer (error 104)
2026-08-07T08:41:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:51 dist-test-slave-h6n0 krb5kdc[11202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:51 dist-test-slave-h6n0 krb5kdc[11202](info): set up 2 sockets
Aug 07 08:41:51 dist-test-slave-h6n0 krb5kdc[11202](info): commencing operation
krb5kdc: starting...
W20260807 08:41:52.553166 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:52 dist-test-slave-h6n0 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:52.697288 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:39607
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:39607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:52.788800 11221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:52.789047 11221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:52.789091 11221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:52.790277 11221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:52.790343 11221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:52.790370 11221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:52.790405 11221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:52.790426 11221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:52.792150 11221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:39607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:52.792600 11221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:52.792817 11221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:52.798688 11230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:52.798693 11228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:52.798693 11227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:52.799099 11221 server_base.cc:1034] running on GCE node
I20260807 08:41:52.799366 11221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:52.799691 11221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:52.800853 11221 hybrid_clock.cc:648] HybridClock initialized: now 1786092112800825 us; error 47 us; skew 500 ppm
Aug 07 08:41:52 dist-test-slave-h6n0 krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:52.803766 11221 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:52.804340 11221 webserver.cc:460] Webserver started at http://127.10.207.62:34525/ using document root <none> and password file <none>
I20260807 08:41:52.804622 11221 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:52.804673 11221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:52.804855 11221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:52.806031 11221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d"
format_stamp: "Formatted at 2026-08-07 08:41:52 on dist-test-slave-h6n0"
server_key: "37c390fd5e063180dd44ab5928f633d1"
server_key_iv: "1bbeecfcc4703a5e6e596b676a151ef8"
server_key_version: "encryptionkey@0"
I20260807 08:41:52.806452 11221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d"
format_stamp: "Formatted at 2026-08-07 08:41:52 on dist-test-slave-h6n0"
server_key: "37c390fd5e063180dd44ab5928f633d1"
server_key_iv: "1bbeecfcc4703a5e6e596b676a151ef8"
server_key_version: "encryptionkey@0"
I20260807 08:41:52.808152 11221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:52.809124 11239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:52.809388 11221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:52.809506 11221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d"
format_stamp: "Formatted at 2026-08-07 08:41:52 on dist-test-slave-h6n0"
server_key: "37c390fd5e063180dd44ab5928f633d1"
server_key_iv: "1bbeecfcc4703a5e6e596b676a151ef8"
server_key_version: "encryptionkey@0"
I20260807 08:41:52.809600 11221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:52.813225 11221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:52.815690 11221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:52.815850 11221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:52.821318 11221 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:39607
I20260807 08:41:52.821377 11309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:39607 every 8 connection(s)
I20260807 08:41:52.821789 11221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:52.823060 11310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:52.823416 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11221
I20260807 08:41:52.823551 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:52.823786 11068 external_mini_cluster.cc:1428] Setting key 1de9bad7742c1baaf76e817302dc19fb
I20260807 08:41:52.826450 11310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d: Bootstrap starting.
I20260807 08:41:52.826762 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:52.824409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57231 (local address 127.10.207.62:39607)
0807 08:41:52.824622 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:52.824631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:52.824661 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:41:52.825274 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:52.825282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:52.825309 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:52.825446 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:52.826194 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:52.826198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:52.826198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:52.826202 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:52.826250 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:52.826262 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:52.826324 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:52.826433 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 08:41:52.826469 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260807 08:41:52.827256 11310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:52.827615 11310 log.cc:824] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:52.827983 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:52.828476 11310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d: No bootstrap required, opened a new log
I20260807 08:41:52.830801 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:52.829617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50397 (local address 127.10.207.62:39607)
0807 08:41:52.829683 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:52.829685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:52.829744 (+ 59us) server_negotiation.cc:407] Connection header received
0807 08:41:52.829799 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:52.829800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:52.829819 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:52.829897 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:52.830493 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:52.830495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:52.830496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:52.830503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:52.830522 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:52.830563 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:52.830654 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:52.830700 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 08:41:52.830718 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:41:52.831238 11310 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39607 } }
I20260807 08:41:52.831447 11310 raft_consensus.cc:374] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:52.831501 11310 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1f3c93bdc94a75a4c01c4d6208dc2d, State: Initialized, Role: FOLLOWER
I20260807 08:41:52.831705 11310 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39607 } }
I20260807 08:41:52.831842 11310 raft_consensus.cc:388] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:52.831910 11310 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:52.831975 11310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:52.832674 11310 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39607 } }
I20260807 08:41:52.832808 11310 leader_election.cc:302] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1f3c93bdc94a75a4c01c4d6208dc2d; no voters:
I20260807 08:41:52.833031 11310 leader_election.cc:288] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:52.833153 11329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:52.833352 11329 raft_consensus.cc:686] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [term 1 LEADER]: Becoming Leader. State: Replica: df1f3c93bdc94a75a4c01c4d6208dc2d, State: Running, Role: LEADER
I20260807 08:41:52.833379 11310 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:52.833468 11329 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39607 } }
I20260807 08:41:52.833923 11332 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1f3c93bdc94a75a4c01c4d6208dc2d. Latest consensus state: current_term: 1 leader_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39607 } } }
I20260807 08:41:52.834028 11332 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:52.834436 11331 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f3c93bdc94a75a4c01c4d6208dc2d" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 39607 } } }
I20260807 08:41:52.834487 11342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:52.835207 11342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:52.834635 11331 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:52.837327 11342 catalog_manager.cc:1347] Generated new cluster ID: 162a8fb1532047079f1aa9307e8049fb
I20260807 08:41:52.837396 11342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:52.848021 11342 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:52.848222 11342 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:52.863216 11342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df1f3c93bdc94a75a4c01c4d6208dc2d: Generated new TSK 0
I20260807 08:41:52.863567 11342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:52.875770 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11221
2026-08-07T08:41:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1615 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:52 dist-test-slave-h6n0 krb5kdc[11357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:52 dist-test-slave-h6n0 krb5kdc[11357](info): set up 2 sockets
Aug 07 08:41:52 dist-test-slave-h6n0 krb5kdc[11357](info): commencing operation
krb5kdc: starting...
W20260807 08:41:54.147360 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:54 dist-test-slave-h6n0 krb5kdc[11357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:54.281657 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45857
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:45857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:54.376475 11376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:54.376734 11376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:54.376773 11376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:54.377923 11376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:54.377992 11376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:54.378019 11376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:54.378050 11376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:54.378077 11376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:54.379717 11376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:45857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:45857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:54.380024 11376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:54.380281 11376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:54.385715 11383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:54.385753 11386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:54.385772 11388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:54.386082 11376 server_base.cc:1034] running on GCE node
I20260807 08:41:54.386370 11376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:54.386777 11376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:54.387954 11376 hybrid_clock.cc:648] HybridClock initialized: now 1786092114387944 us; error 31 us; skew 500 ppm
Aug 07 08:41:54 dist-test-slave-h6n0 krb5kdc[11357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092114, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:54.393805 11376 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:54.394397 11376 webserver.cc:460] Webserver started at http://127.10.207.62:45935/ using document root <none> and password file <none>
I20260807 08:41:54.394754 11376 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:54.394834 11376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:54.394999 11376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:54.396443 11376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "0534826a41af4315a5c5bc705a5e8f5f"
format_stamp: "Formatted at 2026-08-07 08:41:54 on dist-test-slave-h6n0"
server_key: "356278432edd85d8f89f4f1f0ff29e6c"
server_key_iv: "01dd599fc4a649b061c9e8f93353b81a"
server_key_version: "encryptionkey@0"
I20260807 08:41:54.396911 11376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "0534826a41af4315a5c5bc705a5e8f5f"
format_stamp: "Formatted at 2026-08-07 08:41:54 on dist-test-slave-h6n0"
server_key: "356278432edd85d8f89f4f1f0ff29e6c"
server_key_iv: "01dd599fc4a649b061c9e8f93353b81a"
server_key_version: "encryptionkey@0"
I20260807 08:41:54.398618 11376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 08:41:54.399537 11397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:54.399780 11376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:54.399902 11376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "0534826a41af4315a5c5bc705a5e8f5f"
format_stamp: "Formatted at 2026-08-07 08:41:54 on dist-test-slave-h6n0"
server_key: "356278432edd85d8f89f4f1f0ff29e6c"
server_key_iv: "01dd599fc4a649b061c9e8f93353b81a"
server_key_version: "encryptionkey@0"
I20260807 08:41:54.399996 11376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:54.415403 11376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:54.417624 11376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:54.417809 11376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:54.423123 11376 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:45857
I20260807 08:41:54.423295 11470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:45857 every 8 connection(s)
I20260807 08:41:54.423658 11376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:54.424721 11471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:54.427330 11471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f: Bootstrap starting.
I20260807 08:41:54.427968 11471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:54.428284 11471 log.cc:824] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:54.429119 11471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f: No bootstrap required, opened a new log
I20260807 08:41:54.429653 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11376
I20260807 08:41:54.429805 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:54.430027 11068 external_mini_cluster.cc:1428] Setting key 1f48526904f7aff2d2b5653525d8b446
I20260807 08:41:54.431310 11471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0534826a41af4315a5c5bc705a5e8f5f" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45857 } }
I20260807 08:41:54.431494 11471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:54.431535 11471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0534826a41af4315a5c5bc705a5e8f5f, State: Initialized, Role: FOLLOWER
I20260807 08:41:54.431704 11471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0534826a41af4315a5c5bc705a5e8f5f" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45857 } }
I20260807 08:41:54.431804 11471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:54.431841 11471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:54.431896 11471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:54.432906 11471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0534826a41af4315a5c5bc705a5e8f5f" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45857 } }
I20260807 08:41:54.433037 11471 leader_election.cc:302] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0534826a41af4315a5c5bc705a5e8f5f; no voters:
I20260807 08:41:54.433247 11471 leader_election.cc:288] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:54.433274 11481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:54.433403 11481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [term 1 LEADER]: Becoming Leader. State: Replica: 0534826a41af4315a5c5bc705a5e8f5f, State: Running, Role: LEADER
I20260807 08:41:54.433553 11471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:54.433575 11481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0534826a41af4315a5c5bc705a5e8f5f" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45857 } }
I20260807 08:41:54.435461 11486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0534826a41af4315a5c5bc705a5e8f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0534826a41af4315a5c5bc705a5e8f5f" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45857 } } }
I20260807 08:41:54.435575 11488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0534826a41af4315a5c5bc705a5e8f5f. Latest consensus state: current_term: 1 leader_uuid: "0534826a41af4315a5c5bc705a5e8f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0534826a41af4315a5c5bc705a5e8f5f" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 45857 } } }
W20260807 08:41:54.435717 11504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 08:41:54.435722 11488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:54.435830 11504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:54.436118 11486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:54.436329 11506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:54.436630 11506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:41:54 dist-test-slave-h6n0 krb5kdc[11357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:54.439193 11506 catalog_manager.cc:1347] Generated new cluster ID: 1b923869a2ce4259b29d21a9cdc8025b
I20260807 08:41:54.439249 11506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:54.443567 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:54.430682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52829 (local address 127.10.207.62:45857)
0807 08:41:54.430957 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:54.430966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:54.430997 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:41:54.431301 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:54.431308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:54.431364 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:54.431502 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:54.432488 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:54.433750 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:54.435015 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:54.435691 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:54.435830 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:54.439049 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:54.439054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:54.439055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:54.439070 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:54.441640 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:54.442094 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:54.442097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:54.442097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:54.442275 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:54.442580 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:54.442582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:54.442582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:54.442922 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:54.443101 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:54.443195 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 08:41:54.443348 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":132,"threads_started":1}
I20260807 08:41:54.444862 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:54.454044 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:54.446572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53283 (local address 127.10.207.62:45857)
0807 08:41:54.446616 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:54.446618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:54.446768 (+ 150us) server_negotiation.cc:407] Connection header received
0807 08:41:54.446792 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:54.446794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:54.446805 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:54.446866 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:54.447506 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:54.448294 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:54.449280 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:54.449628 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:54.449706 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:54.451156 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:54.451157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:54.451158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:54.451165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:54.452809 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:54.453206 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:54.453207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:54.453207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:54.453268 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:54.453490 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:54.453491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:54.453491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:54.453631 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:54.453732 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:54.453867 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:41:54.453950 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:41:54.456578 11506 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:54.457360 11506 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:54.463692 11506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0534826a41af4315a5c5bc705a5e8f5f: Generated new TSK 0
I20260807 08:41:54.463971 11506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:54.467072 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11376
2026-08-07T08:41:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:54 dist-test-slave-h6n0 krb5kdc[11520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:54 dist-test-slave-h6n0 krb5kdc[11520](info): set up 2 sockets
Aug 07 08:41:54 dist-test-slave-h6n0 krb5kdc[11520](info): commencing operation
krb5kdc: starting...
W20260807 08:41:55.744488 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:55 dist-test-slave-h6n0 krb5kdc[11520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:55.882121 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:40953
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:40953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:55.973979 11535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:55.974232 11535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:55.974267 11535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:55.975508 11535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:55.975580 11535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:55.975615 11535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:55.975647 11535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:55.975679 11535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:55.977187 11535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:40953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:40953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:55.977476 11535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:55.977736 11535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:55.983078 11543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:55.983076 11541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:55.983240 11545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:55.983541 11535 server_base.cc:1034] running on GCE node
I20260807 08:41:55.983875 11535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:55.984208 11535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:55.985380 11535 hybrid_clock.cc:648] HybridClock initialized: now 1786092115985349 us; error 51 us; skew 500 ppm
Aug 07 08:41:55 dist-test-slave-h6n0 krb5kdc[11520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:55.988442 11535 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:55.988927 11535 webserver.cc:460] Webserver started at http://127.10.207.62:34089/ using document root <none> and password file <none>
I20260807 08:41:55.989167 11535 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:55.989216 11535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:55.989318 11535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:55.990489 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "1308e92231424a9c9e7a0168086041db"
format_stamp: "Formatted at 2026-08-07 08:41:55 on dist-test-slave-h6n0"
server_key: "80685e934a8f60e868911ceed0449978"
server_key_iv: "05150b1d7dca10717b851ba4154ba0c0"
server_key_version: "encryptionkey@0"
I20260807 08:41:55.990900 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "1308e92231424a9c9e7a0168086041db"
format_stamp: "Formatted at 2026-08-07 08:41:55 on dist-test-slave-h6n0"
server_key: "80685e934a8f60e868911ceed0449978"
server_key_iv: "05150b1d7dca10717b851ba4154ba0c0"
server_key_version: "encryptionkey@0"
I20260807 08:41:55.992245 11535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:55.993036 11555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:55.993289 11535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:55.993413 11535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "1308e92231424a9c9e7a0168086041db"
format_stamp: "Formatted at 2026-08-07 08:41:55 on dist-test-slave-h6n0"
server_key: "80685e934a8f60e868911ceed0449978"
server_key_iv: "05150b1d7dca10717b851ba4154ba0c0"
server_key_version: "encryptionkey@0"
I20260807 08:41:55.993491 11535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:56.002435 11535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:56.004598 11535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:56.004755 11535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:56.009127 11535 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:40953
I20260807 08:41:56.009205 11633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:40953 every 8 connection(s)
I20260807 08:41:56.009552 11535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:56.010597 11635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:56.013288 11635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db: Bootstrap starting.
I20260807 08:41:56.014017 11635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:56.014387 11635 log.cc:824] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:56.015261 11635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db: No bootstrap required, opened a new log
I20260807 08:41:56.017079 11635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1308e92231424a9c9e7a0168086041db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40953 } }
I20260807 08:41:56.017258 11635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:56.017287 11635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1308e92231424a9c9e7a0168086041db, State: Initialized, Role: FOLLOWER
I20260807 08:41:56.017462 11635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1308e92231424a9c9e7a0168086041db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40953 } }
I20260807 08:41:56.017558 11635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:56.017585 11635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:56.017639 11635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:56.018436 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11535
I20260807 08:41:56.018612 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:56.018839 11068 external_mini_cluster.cc:1428] Setting key aa4274b960a54ac242bb36c4fa6eb352
I20260807 08:41:56.018923 11635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1308e92231424a9c9e7a0168086041db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40953 } }
I20260807 08:41:56.019035 11635 leader_election.cc:302] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1308e92231424a9c9e7a0168086041db; no voters:
I20260807 08:41:56.019248 11635 leader_election.cc:288] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:56.019454 11639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:56.019585 11635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:56.019690 11639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [term 1 LEADER]: Becoming Leader. State: Replica: 1308e92231424a9c9e7a0168086041db, State: Running, Role: LEADER
I20260807 08:41:56.019853 11639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1308e92231424a9c9e7a0168086041db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40953 } }
I20260807 08:41:56.020434 11642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1308e92231424a9c9e7a0168086041db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1308e92231424a9c9e7a0168086041db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40953 } } }
I20260807 08:41:56.020545 11642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:56.020898 11646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1308e92231424a9c9e7a0168086041db. Latest consensus state: current_term: 1 leader_uuid: "1308e92231424a9c9e7a0168086041db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1308e92231424a9c9e7a0168086041db" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 40953 } } }
I20260807 08:41:56.021006 11646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:56.022241 11659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:56.022344 11659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:56.022452 11653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:56.022646 11653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:56.024554 11653 catalog_manager.cc:1347] Generated new cluster ID: f0a92a1339544133850b0b2b2cc74113
I20260807 08:41:56.024621 11653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:56.035652 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:56.019459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44599 (local address 127.10.207.62:40953)
0807 08:41:56.019931 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:56.019941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:56.019975 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:41:56.020118 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:56.020128 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:56.020203 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:56.020388 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:56.020709 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:56.022248 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:56.023246 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:56.023878 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:56.024013 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:56.034831 (+ 10818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:56.034836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:56.034837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:56.035077 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:56.035121 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:56.035137 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:56.035233 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:56.035341 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 08:41:56.035426 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":123,"threads_started":1}
I20260807 08:41:56.036793 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:56.037954 11653 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:56.038729 11653 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:56.047227 11653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1308e92231424a9c9e7a0168086041db: Generated new TSK 0
I20260807 08:41:56.047468 11653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:56.049552 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:56.045586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35001 (local address 127.10.207.62:40953)
0807 08:41:56.045668 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:56.045670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:56.045823 (+ 153us) server_negotiation.cc:407] Connection header received
0807 08:41:56.045855 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:56.045856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:56.045881 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:56.045953 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:56.046208 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:56.047061 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:56.048033 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:56.048299 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:56.048382 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:56.049272 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:56.049273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:56.049274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:56.049279 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:56.049296 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:56.049307 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:56.049371 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:56.049436 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:41:56.049465 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 08:41:56.051944 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11535
2026-08-07T08:41:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1579 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:56 dist-test-slave-h6n0 krb5kdc[11673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:56 dist-test-slave-h6n0 krb5kdc[11673](info): set up 2 sockets
Aug 07 08:41:56 dist-test-slave-h6n0 krb5kdc[11673](info): commencing operation
krb5kdc: starting...
W20260807 08:41:57.309103 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:57 dist-test-slave-h6n0 krb5kdc[11673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:57.439177 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:46377
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:37171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:46377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:57.532521 11692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:57.532770 11692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:57.532794 11692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:57.533987 11692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:57.534036 11692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:57.534051 11692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:57.534065 11692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:57.534076 11692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:57.535749 11692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:37171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:46377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:57.536048 11692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:57.536314 11692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:57.542218 11697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:57.542167 11700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:57.542372 11692 server_base.cc:1034] running on GCE node
W20260807 08:41:57.542156 11698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:57.542788 11692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:57.543165 11692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:57.544324 11692 hybrid_clock.cc:648] HybridClock initialized: now 1786092117544303 us; error 41 us; skew 500 ppm
Aug 07 08:41:57 dist-test-slave-h6n0 krb5kdc[11673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092117, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:57.547188 11692 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:57.547742 11692 webserver.cc:460] Webserver started at http://127.10.207.62:39565/ using document root <none> and password file <none>
I20260807 08:41:57.547995 11692 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:57.548087 11692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:57.548254 11692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:57.549464 11692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "345942f15a314330b35aaa60f4170f10"
format_stamp: "Formatted at 2026-08-07 08:41:57 on dist-test-slave-h6n0"
server_key: "c8eacb166bb27baeb2fabdc34f727961"
server_key_iv: "bc21d307d2fa4ebfd2bc3059b19f9f27"
server_key_version: "encryptionkey@0"
I20260807 08:41:57.549888 11692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "345942f15a314330b35aaa60f4170f10"
format_stamp: "Formatted at 2026-08-07 08:41:57 on dist-test-slave-h6n0"
server_key: "c8eacb166bb27baeb2fabdc34f727961"
server_key_iv: "bc21d307d2fa4ebfd2bc3059b19f9f27"
server_key_version: "encryptionkey@0"
I20260807 08:41:57.551491 11692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:41:57.552358 11707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:57.552641 11692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:57.552785 11692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "345942f15a314330b35aaa60f4170f10"
format_stamp: "Formatted at 2026-08-07 08:41:57 on dist-test-slave-h6n0"
server_key: "c8eacb166bb27baeb2fabdc34f727961"
server_key_iv: "bc21d307d2fa4ebfd2bc3059b19f9f27"
server_key_version: "encryptionkey@0"
I20260807 08:41:57.552874 11692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:57.557134 11692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:57.559437 11692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:57.559631 11692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:57.564967 11692 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:46377
I20260807 08:41:57.565023 11793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:46377 every 8 connection(s)
I20260807 08:41:57.565564 11692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:57.566869 11794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:57.569491 11794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10: Bootstrap starting.
I20260807 08:41:57.570484 11794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:57.570859 11794 log.cc:824] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:57.571731 11794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10: No bootstrap required, opened a new log
I20260807 08:41:57.573678 11794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345942f15a314330b35aaa60f4170f10" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46377 } }
I20260807 08:41:57.573877 11794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:41:57.573941 11794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345942f15a314330b35aaa60f4170f10, State: Initialized, Role: FOLLOWER
I20260807 08:41:57.574117 11794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345942f15a314330b35aaa60f4170f10" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46377 } }
I20260807 08:41:57.574208 11794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:41:57.574254 11794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:41:57.574303 11794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:57.575446 11794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345942f15a314330b35aaa60f4170f10" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46377 } }
I20260807 08:41:57.575615 11794 leader_election.cc:302] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345942f15a314330b35aaa60f4170f10; no voters:
I20260807 08:41:57.575739 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11692
I20260807 08:41:57.575860 11794 leader_election.cc:288] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:41:57.575933 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:57.575944 11799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:57.576121 11799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [term 1 LEADER]: Becoming Leader. State: Replica: 345942f15a314330b35aaa60f4170f10, State: Running, Role: LEADER
I20260807 08:41:57.576184 11794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:57.576202 11068 external_mini_cluster.cc:1428] Setting key e2c0e13c4198518498d097e96558534b
I20260807 08:41:57.576292 11799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345942f15a314330b35aaa60f4170f10" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46377 } }
I20260807 08:41:57.577008 11802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345942f15a314330b35aaa60f4170f10. Latest consensus state: current_term: 1 leader_uuid: "345942f15a314330b35aaa60f4170f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345942f15a314330b35aaa60f4170f10" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46377 } } }
I20260807 08:41:57.577188 11802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:57.577442 11800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "345942f15a314330b35aaa60f4170f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345942f15a314330b35aaa60f4170f10" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 46377 } } }
I20260807 08:41:57.577603 11800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:57.577812 11813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:57.578449 11813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:57.578780 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:57.576819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48423 (local address 127.10.207.62:46377)
0807 08:41:57.577028 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:57.577037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:57.577073 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:41:57.577202 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:57.577210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:57.577276 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:57.577421 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:57.578247 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:57.578251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:57.578251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:57.578257 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:57.578308 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:57.578326 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:57.578370 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:57.578504 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 08:41:57.578582 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260807 08:41:57.580292 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:57.580551 11813 catalog_manager.cc:1347] Generated new cluster ID: 65711f08a59b4a12acfba039186482b0
I20260807 08:41:57.580596 11813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:57.583280 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:57.582078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40031 (local address 127.10.207.62:46377)
0807 08:41:57.582118 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:57.582120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:57.582204 (+ 84us) server_negotiation.cc:407] Connection header received
0807 08:41:57.582293 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:57.582294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:57.582301 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:57.582367 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:57.583020 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:57.583021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:57.583021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:41:57.583026 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:57.583044 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:41:57.583049 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:57.583112 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:57.583196 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 08:41:57.583216 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:41:57.600852 11813 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:57.601073 11813 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:57.610265 11813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 345942f15a314330b35aaa60f4170f10: Generated new TSK 0
I20260807 08:41:57.610478 11813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:57.620975 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11692
2026-08-07T08:41:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1570 ms)
[----------] 18 tests from AuthTokenIssuingTest (28748 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:41:57 dist-test-slave-h6n0 krb5kdc[11838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:41:57 dist-test-slave-h6n0 krb5kdc[11838](info): set up 2 sockets
Aug 07 08:41:57 dist-test-slave-h6n0 krb5kdc[11838](info): commencing operation
krb5kdc: starting...
W20260807 08:41:58.885546 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:41:58 dist-test-slave-h6n0 krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:41:59.017234 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41611
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:41611,127.10.207.61:39917,127.10.207.60:42325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:41:59.110299 11855 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:41:59.110639 11855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:59.110683 11855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:59.110728 11855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:59.111913 11855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:59.111984 11855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:59.112008 11855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:59.112030 11855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:59.112061 11855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:59.113668 11855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:41611,127.10.207.61:39917,127.10.207.60:42325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:41611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:59.113983 11855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:59.114259 11855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:59.119791 11864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:59.119850 11861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:59.119952 11867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:59.120066 11855 server_base.cc:1034] running on GCE node
I20260807 08:41:59.120532 11855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:59.121066 11855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:59.122349 11855 hybrid_clock.cc:648] HybridClock initialized: now 1786092119122321 us; error 39 us; skew 500 ppm
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:59.125465 11855 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:41:59.125991 11855 webserver.cc:460] Webserver started at http://127.10.207.62:40397/ using document root <none> and password file <none>
I20260807 08:41:59.126246 11855 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:59.126317 11855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:59.126482 11855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:59.127655 11855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "e44c49549bf74fe8a8cf78ccf4e63176"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "75f08c2614f060ec19eca0b931531935"
server_key_iv: "8ef2de26b72b41cc655697f7f937bf77"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.128079 11855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "e44c49549bf74fe8a8cf78ccf4e63176"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "75f08c2614f060ec19eca0b931531935"
server_key_iv: "8ef2de26b72b41cc655697f7f937bf77"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.129496 11855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:41:59.130299 11877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:59.130574 11855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:41:59.130695 11855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "e44c49549bf74fe8a8cf78ccf4e63176"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "75f08c2614f060ec19eca0b931531935"
server_key_iv: "8ef2de26b72b41cc655697f7f937bf77"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.130779 11855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:59.152328 11855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:59.154219 11855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:59.154393 11855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:59.158886 11855 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:41611
I20260807 08:41:59.159004 11948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:41611 every 8 connection(s)
I20260807 08:41:59.159322 11855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:41:59.160537 11949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:59.162444 11949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:41:59.163398 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11855
I20260807 08:41:59.163528 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:41:59.163749 11068 external_mini_cluster.cc:1428] Setting key 5fdaa60c3eda4ac633c68a931b79331f
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.207.61@KRBTEST.COM; defaulting to no policy
I20260807 08:41:59.175048 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.163388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57114 (local address 127.10.207.62:41611)
0807 08:41:59.163630 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.163636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.163674 (+ 38us) server_negotiation.cc:407] Connection header received
0807 08:41:59.164617 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.164620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.164641 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.164744 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.165652 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.166764 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.167996 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.168358 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.168620 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.171497 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.171504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.171505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.171511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.173244 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.173767 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.173770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.173770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.173850 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.174141 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.174141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.174142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.174352 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.174421 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.174707 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.174860 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":207,"thread_start_us":84,"threads_started":1}
I20260807 08:41:59.175168 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.163244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41611 (local address 127.0.0.1:57114)
0807 08:41:59.163492 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.163552 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.164371 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.164785 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.164789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.165213 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.165496 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.165502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.166870 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.166871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.167891 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.167894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.168525 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.168526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.168713 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.169373 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.169382 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.171377 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.173353 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.173362 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.173364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.173655 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.173931 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.173931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.173932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.174057 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.174438 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.174446 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.174557 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.174663 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.174971 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":116,"thread_start_us":142,"threads_started":1}
I20260807 08:41:59.176129 11949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:41:59.176529 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.176400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60932)
0807 08:41:59.176453 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.176501 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:41:59.176671 11883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
I20260807 08:41:59.177163 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.177067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60938)
0807 08:41:59.177117 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.177141 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:41:59.177279 11949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:39917: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab.
Entry for principal kudu/127.10.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab.
I20260807 08:41:59.193569 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:39917
--webserver_interface=127.10.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab
--principal=kudu/127.10.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:41611,127.10.207.61:39917,127.10.207.60:42325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:41:59.226575 11949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } attempt: 1
I20260807 08:41:59.227178 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.227021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60942)
0807 08:41:59.227099 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.227143 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 08:41:59.227720 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.227624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60946)
0807 08:41:59.227676 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.227703 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 08:41:59.227846 11949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:39917: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
W20260807 08:41:59.281369 11961 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:41:59.281611 11961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:59.281643 11961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:59.281693 11961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:59.282855 11961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:59.282913 11961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:59.283229 11961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:59.283308 11961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:59.283334 11961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:59.284868 11961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:41611,127.10.207.61:39917,127.10.207.60:42325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab
--principal=kudu/127.10.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:39917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:59.285142 11961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:59.285389 11961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:59.290257 11974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:59.290299 11969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:59.290299 11967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:59.290766 11961 server_base.cc:1034] running on GCE node
I20260807 08:41:59.291047 11961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:59.291368 11961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:59.292526 11961 hybrid_clock.cc:648] HybridClock initialized: now 1786092119292488 us; error 58 us; skew 500 ppm
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:59.295392 11961 init.cc:377] Logged in from keytab as kudu/127.10.207.61@KRBTEST.COM (short username kudu)
I20260807 08:41:59.295914 11961 webserver.cc:460] Webserver started at http://127.10.207.61:41317/ using document root <none> and password file <none>
I20260807 08:41:59.296008 11949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } attempt: 2
I20260807 08:41:59.296233 11961 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:59.296310 11961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:59.296463 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.296320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60958)
0807 08:41:59.296392 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.296423 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 08:41:59.296469 11961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:59.296885 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.296762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60972)
0807 08:41:59.296826 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.296859 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:41:59.297008 11949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:39917: Network error: Client connection negotiation failed: client connection to 127.10.207.61:39917: connect: Connection refused (error 111)
I20260807 08:41:59.297816 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data/instance:
uuid: "d61e4413174047a280837d66503c989f"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "b4cd4432921795d77f61dfffdfc86855"
server_key_iv: "f289a32556f1f1bc129cb7c9845726a7"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.298233 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/wal/instance:
uuid: "d61e4413174047a280837d66503c989f"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "b4cd4432921795d77f61dfffdfc86855"
server_key_iv: "f289a32556f1f1bc129cb7c9845726a7"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.299783 11961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 08:41:59.300714 11982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:59.301005 11961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:59.301156 11961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/wal
uuid: "d61e4413174047a280837d66503c989f"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "b4cd4432921795d77f61dfffdfc86855"
server_key_iv: "f289a32556f1f1bc129cb7c9845726a7"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.301234 11961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:59.308569 11961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:59.310858 11961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:59.311012 11961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:59.315788 11961 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.61:39917
I20260807 08:41:59.315876 12059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.61:39917 every 8 connection(s)
I20260807 08:41:59.316259 11961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
I20260807 08:41:59.317327 12061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:59.319080 12061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:41:59.320067 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 11961
I20260807 08:41:59.320169 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-1/wal/instance
I20260807 08:41:59.320397 11068 external_mini_cluster.cc:1428] Setting key 9ee76e18b83dbffd554bf5d5f5e2427f
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.207.60@KRBTEST.COM; defaulting to no policy
I20260807 08:41:59.330564 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.319914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.10.207.62:41611)
0807 08:41:59.320001 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.320005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.320325 (+ 320us) server_negotiation.cc:407] Connection header received
0807 08:41:59.321025 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.321026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.321038 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.321110 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.321997 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.322616 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.323955 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.324214 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.324288 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.327175 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.327176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.327177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.327185 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.328860 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.329483 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.329484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.329484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.329560 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.329860 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.329861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.329862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.329986 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.330062 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.330351 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.330445 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:41:59.330765 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.319863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41611 (local address 127.0.0.1:57122)
0807 08:41:59.320147 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.320203 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.320894 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.321201 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.321205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.321600 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.321888 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.321894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.322729 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.322731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.323785 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.323790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.324346 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.324348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.324514 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.325098 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.325109 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.327067 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.329007 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.329017 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.329019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.329370 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.329640 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.329641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.329641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.329771 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.330068 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.330075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.330216 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.330327 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.330505 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":134,"threads_started":1}
I20260807 08:41:59.331565 12061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.60@KRBTEST.COM" created.
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:41:59.340827 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.331952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60978)
0807 08:41:59.332041 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.332051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.332085 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.332390 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.332391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.332577 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.332657 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.332659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.333893 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.333894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.334493 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.334495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.335133 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.335134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.335213 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.335677 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.335682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.337135 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.339572 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.339572 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.339573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.339758 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.340068 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.340069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.340069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.340132 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.340521 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.340522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.340567 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.340644 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.340763 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 08:41:59.341020 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.332030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.10.207.61:39917)
0807 08:41:59.332210 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.332215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.332237 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:41:59.332261 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.332263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.332277 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.332370 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.332744 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.333771 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.334702 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.335034 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.335134 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.337246 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.337261 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.337262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.337268 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.339449 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.339870 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.339875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.339875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.339951 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.340216 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.340217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.340217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.340427 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.340536 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.340702 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.340877 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":148,"thread_start_us":118,"threads_started":1}
I20260807 08:41:59.341754 12061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:41:59.342094 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.341968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38702)
0807 08:41:59.342019 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.342065 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:41:59.342232 11989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
I20260807 08:41:59.342691 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.342569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38718)
0807 08:41:59.342616 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.342656 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:41:59.342869 12061 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:42325: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab.
Entry for principal kudu/127.10.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab.
I20260807 08:41:59.349709 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:42325
--webserver_interface=127.10.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab
--principal=kudu/127.10.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:41611,127.10.207.61:39917,127.10.207.60:42325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:41:59.388135 11949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } attempt: 3
I20260807 08:41:59.392151 12061 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } attempt: 1
I20260807 08:41:59.392874 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.392735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38728)
0807 08:41:59.392798 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.392840 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 08:41:59.393239 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.393159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38742)
0807 08:41:59.393189 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.393215 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 08:41:59.393366 12061 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:42325: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:41:59.400275 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.388523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:60994)
0807 08:41:59.388645 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.388667 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.388838 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.389086 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.389088 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.389457 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.389650 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.389653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.390717 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.390745 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.391429 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.391431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.391947 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.391947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.392015 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.395507 (+ 3492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.395513 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.397124 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.399104 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.399106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.399106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.399304 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.399591 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.399592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.399592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.399646 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.399920 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.399921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.399978 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.400050 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.400168 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:41:59.400303 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.388654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.10.207.61:39917)
0807 08:41:59.388727 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.388729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.388793 (+ 64us) server_negotiation.cc:407] Connection header received
0807 08:41:59.388942 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.388943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.388957 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.389028 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.389784 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.390628 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.391521 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.391862 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.391945 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.397226 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.397228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.397228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.397237 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.398999 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.399416 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.399419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.399419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.399494 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.399711 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.399712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.399713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.399846 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.399935 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.400130 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.400218 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:41:59.401115 11949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:41:59.401521 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.401408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38746)
0807 08:41:59.401462 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.401493 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 08:41:59.401954 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.401890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38754)
0807 08:41:59.401915 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.401939 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 08:41:59.402062 11949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:42325: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
I20260807 08:41:59.433211 11949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } attempt: 1
I20260807 08:41:59.433751 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.433610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38768)
0807 08:41:59.433684 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.433720 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 08:41:59.434150 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.434054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38776)
0807 08:41:59.434099 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.434124 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 08:41:59.434248 11949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:42325: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
W20260807 08:41:59.439003 12074 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:41:59.439232 12074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:41:59.439273 12074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:41:59.439308 12074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:41:59.440393 12074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:41:59.440465 12074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:41:59.440495 12074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:41:59.440526 12074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:41:59.440550 12074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:41:59.442045 12074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:41611,127.10.207.61:39917,127.10.207.60:42325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab
--principal=kudu/127.10.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:42325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:41:59.442315 12074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:41:59.442577 12074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:41:59.447516 12084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:59.447597 12079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:41:59.447526 12081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:41:59.447781 12074 server_base.cc:1034] running on GCE node
I20260807 08:41:59.448184 12074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:41:59.448477 12074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:41:59.449648 12074 hybrid_clock.cc:648] HybridClock initialized: now 1786092119449634 us; error 29 us; skew 500 ppm
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:41:59.452404 12074 init.cc:377] Logged in from keytab as kudu/127.10.207.60@KRBTEST.COM (short username kudu)
I20260807 08:41:59.452852 12074 webserver.cc:460] Webserver started at http://127.10.207.60:36847/ using document root <none> and password file <none>
I20260807 08:41:59.453037 12074 fs_manager.cc:359] Metadata directory not provided
I20260807 08:41:59.453081 12074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:41:59.453168 12074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:41:59.454128 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data/instance:
uuid: "a0140423275541afb386463a5d78a76d"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "11732a798f354ed41a250e1af5729758"
server_key_iv: "7582f131d68d32589ae1c38876620942"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.454418 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/wal/instance:
uuid: "a0140423275541afb386463a5d78a76d"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "11732a798f354ed41a250e1af5729758"
server_key_iv: "7582f131d68d32589ae1c38876620942"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.455799 12074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:59.456612 12096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:41:59.456873 12074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:41:59.456979 12074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/wal
uuid: "a0140423275541afb386463a5d78a76d"
format_stamp: "Formatted at 2026-08-07 08:41:59 on dist-test-slave-h6n0"
server_key: "11732a798f354ed41a250e1af5729758"
server_key_iv: "7582f131d68d32589ae1c38876620942"
server_key_version: "encryptionkey@0"
I20260807 08:41:59.457064 12074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:41:59.461517 12061 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } attempt: 2
I20260807 08:41:59.461990 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.461856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38780)
0807 08:41:59.461922 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.461960 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 08:41:59.462447 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.462358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38792)
0807 08:41:59.462406 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.462432 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 08:41:59.462606 12061 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:42325: Network error: Client connection negotiation failed: client connection to 127.10.207.60:42325: connect: Connection refused (error 111)
I20260807 08:41:59.470042 12074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:41:59.472143 12074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:41:59.472317 12074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:41:59.477020 12074 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.60:42325
I20260807 08:41:59.477079 12163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.60:42325 every 8 connection(s)
I20260807 08:41:59.477398 12074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
I20260807 08:41:59.478616 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:41:59.480492 12165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:41:59.485569 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12074
I20260807 08:41:59.485656 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091830136701-11068-0/minicluster-data/master-2/wal/instance
I20260807 08:41:59.485939 11068 external_mini_cluster.cc:1428] Setting key 3b590053a51f64fe300f2430df58bd72
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:41:59.492059 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.481313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.10.207.62:41611)
0807 08:41:59.481442 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.481449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.481685 (+ 236us) server_negotiation.cc:407] Connection header received
0807 08:41:59.482280 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.482282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.482294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.482358 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.483224 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.483749 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.484841 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.485075 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.485149 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.488508 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.488510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.488510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.488518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.490200 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.490787 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.490788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.490789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.490857 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.491196 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.491197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.491197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.491311 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.491364 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.491842 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.491944 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 08:41:59.492267 12167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.481247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:41611 (local address 127.0.0.1:57138)
0807 08:41:59.481523 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.481566 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.482151 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.482436 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.482440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.482825 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.483052 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.483057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.483848 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.483849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.484700 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.484702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.485215 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.485216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.485333 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.485936 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.485945 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.488406 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.490293 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.490301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.490304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.490702 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.490946 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.490946 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.490947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.491108 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.491386 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.491394 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.491642 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.491778 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.491960 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":142,"threads_started":1}
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:41:59.493054 12165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:41:59.496673 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.486423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.10.207.60:42325)
0807 08:41:59.486599 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.486603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.486761 (+ 158us) server_negotiation.cc:407] Connection header received
0807 08:41:59.486796 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.486799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.486823 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.486904 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.487655 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.488559 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.489643 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.490159 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.490248 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.493621 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.493628 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.493628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.493634 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.495280 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.495714 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.495718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.495718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.495787 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.496057 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.496058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.496058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.496224 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.496347 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.496466 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.496534 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:41:59.500329 12167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.493365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:39917 (local address 127.0.0.1:32774)
0807 08:41:59.493525 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.493538 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.493605 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.493740 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.493741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.493883 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.493933 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.493935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.494619 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.494620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.495112 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.495113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.495519 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.495520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.495604 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.496004 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.496009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.497520 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.499277 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.499278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.499279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.499449 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.499671 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.499672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.499673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.499734 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.499996 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.499997 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.500042 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.500127 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.500230 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 08:41:59.500329 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.493460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.10.207.61:39917)
0807 08:41:59.493529 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.493531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.493584 (+ 53us) server_negotiation.cc:407] Connection header received
0807 08:41:59.493673 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.493674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.493687 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.493738 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.494019 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.494495 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.495194 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.495430 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.495518 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.497668 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.497670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.497670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.497680 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.499189 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.499526 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.499527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.499528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.499586 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.499795 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.499796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.499797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.499919 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.499995 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.500156 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.500231 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:41:59.500949 12165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:41:59.506646 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.497608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.10.207.62:41611)
0807 08:41:59.497951 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.497953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.497962 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:41:59.497981 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.497982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.497992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.498055 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.498550 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.499056 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.500023 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.500238 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.500335 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.503338 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.503339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.503340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.503346 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.505085 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.505461 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.505463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.505463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.505526 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.505764 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.505765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.505766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.505924 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.506067 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.506373 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.506480 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260807 08:41:59.508579 12167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.501263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38802)
0807 08:41:59.501307 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.501316 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.501354 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.501543 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.501544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.501663 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.501692 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.501693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.502483 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.502484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.503164 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.503167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.503705 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.503706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.503792 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.504254 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.504259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.505462 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.507320 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.507321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.507321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.507556 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.507810 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.507810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.507810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.507866 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.508190 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.508191 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.508227 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.508329 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.508463 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:41:59.508728 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.501350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38802 (local address 127.10.207.60:42325)
0807 08:41:59.501399 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.501401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.501411 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:41:59.501434 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.501434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.501447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.501534 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.501777 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.502394 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.503262 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.503587 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.503690 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.505535 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.505536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.505537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.505543 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.507233 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.507665 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.507666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.507666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.507727 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.507943 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.507944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.507944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.508110 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.508188 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.508371 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.508513 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":28}
I20260807 08:41:59.509394 11949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } attempt: 2
I20260807 08:41:59.511049 12165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: Bootstrap starting.
I20260807 08:41:59.512151 12165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:59.512554 12165 log.cc:824] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:41:59.514737 12165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: No bootstrap required, opened a new log
I20260807 08:41:59.516743 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.507475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.10.207.61:39917)
0807 08:41:59.507522 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.507525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.507534 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:41:59.507557 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.507558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.507569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.507631 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.508278 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.509378 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.510265 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.510631 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.510712 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.513506 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.513507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.513508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.513516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.515280 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.515739 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.515741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.515741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.515816 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.516110 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.516111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.516111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.516245 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.516347 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.516541 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.516629 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:41:59.517416 12165 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.517851 12165 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0140423275541afb386463a5d78a76d, State: Initialized, Role: FOLLOWER
I20260807 08:41:59.517947 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.509659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38810)
0807 08:41:59.509726 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.509741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.509789 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.509917 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.509918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.510074 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.510130 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.510133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.510978 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.510979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.511839 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.511843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.512430 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.512431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.512497 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.512930 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.512935 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.514740 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.516655 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.516656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.516657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.516886 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.517184 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.517185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.517186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.517266 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.517608 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.517609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.517654 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.517729 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.517832 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:41:59.517954 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.509774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.10.207.60:42325)
0807 08:41:59.509805 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.509807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.509815 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:41:59.509840 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.509841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.509848 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.509915 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.510222 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.510866 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.511953 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.512322 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.512414 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.514886 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.514887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.514888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.514895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.516540 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.516985 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.516986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.516986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.517043 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.517353 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.517354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.517354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.517518 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.517616 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.517784 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.517882 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:41:59.517963 12165 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.518338 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:41:59.518417 12165 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:59.518378 12172 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.518523 12172 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:41:59.520596 11949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176: Bootstrap starting.
I20260807 08:41:59.519727 11916 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e44c49549bf74fe8a8cf78ccf4e63176"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57150
W20260807 08:41:59.520746 12188 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:59.520804 12188 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:59.521397 11949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:59.521718 11949 log.cc:824] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:59.522583 11949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176: No bootstrap required, opened a new log
I20260807 08:41:59.520881 12074 master_runner.cc:186] Error getting master registration for 127.10.207.62:41611: OK, instance_id { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" instance_seqno: 1786092119157586 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:41:59.523633 12074 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:41:59.523932 11949 raft_consensus.cc:348] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.524067 11949 raft_consensus.cc:729] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44c49549bf74fe8a8cf78ccf4e63176, State: Initialized, Role: FOLLOWER
I20260807 08:41:59.524163 11949 consensus_queue.cc:260] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.524524 11949 sys_catalog.cc:564] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:59.524607 11916 raft_consensus.cc:482] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:41:59.524696 11916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:59.525760 11916 raft_consensus.cc:504] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.526590 12130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a0140423275541afb386463a5d78a76d"
I20260807 08:41:59.526736 12130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:59.527572 12130 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44c49549bf74fe8a8cf78ccf4e63176 in term 1.
I20260807 08:41:59.527868 11884 leader_election.cc:302] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0140423275541afb386463a5d78a76d, e44c49549bf74fe8a8cf78ccf4e63176; no voters:
I20260807 08:41:59.526058 11916 leader_election.cc:288] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [CANDIDATE]: Term 1 election: Requested vote from peers d61e4413174047a280837d66503c989f (127.10.207.61:39917), a0140423275541afb386463a5d78a76d (127.10.207.60:42325)
I20260807 08:41:59.526304 12191 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.528165 12191 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:41:59.528246 12191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:41:59.528307 12191 raft_consensus.cc:686] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [term 1 LEADER]: Becoming Leader. State: Replica: e44c49549bf74fe8a8cf78ccf4e63176, State: Running, Role: LEADER
I20260807 08:41:59.528402 12191 consensus_queue.cc:237] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.528867 12191 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e44c49549bf74fe8a8cf78ccf4e63176. Latest consensus state: current_term: 1 leader_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.528970 12191 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:59.526679 12206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:59.530228 12206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:59.530510 12221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:41:59.530896 12221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:41:59.530115 11855 master_runner.cc:186] Error getting master registration for 127.10.207.61:39917: OK, instance_id { permanent_uuid: "d61e4413174047a280837d66503c989f" instance_seqno: 1786092119314303 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:41:59.533208 11855 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:41:59.538180 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.529275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.10.207.60:42325)
0807 08:41:59.529313 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.529315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.529680 (+ 365us) server_negotiation.cc:407] Connection header received
0807 08:41:59.529800 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.529801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.529812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.529845 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.530956 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.531728 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.532900 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.533187 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.533232 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.535074 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.535076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.535076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.535083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.536889 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.537289 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.537291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.537291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.537349 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.537546 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.537547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.537547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.537685 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.537776 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.537997 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.538078 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:41:59.546060 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.529896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57160 (local address 127.10.207.62:41611)
0807 08:41:59.529933 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.529935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.538672 (+ 8737us) server_negotiation.cc:407] Connection header received
0807 08:41:59.538776 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.538778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.538789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.538820 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.539425 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.540176 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.541198 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.541496 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.541535 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.543028 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.543029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.543030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.543037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.544733 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.545143 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.545145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.545145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.545203 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.545417 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.545418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.545419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.545545 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.545651 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.545880 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.545959 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:41:59.548844 12130 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [term 1 FOLLOWER]: Refusing update from remote peer e44c49549bf74fe8a8cf78ccf4e63176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:59.549306 12208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:59.551059 12172 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e44c49549bf74fe8a8cf78ccf4e63176. Latest consensus state: current_term: 1 leader_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.551162 12172 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:41:59.553666 12191 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.553754 12191 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: This master's current role is: LEADER
I20260807 08:41:59.553750 12061 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } attempt: 3
I20260807 08:41:59.553823 12223 mvcc.cc:205] Tried to move back new op lower bound from 7315833321667252224 to 7315833321589211136. Current Snapshot: MvccSnapshot[applied={T|T < 7315833321667252224}]
I20260807 08:41:59.553879 12172 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.553982 12172 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:41:59.555876 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.529698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.10.207.61:39917)
0807 08:41:59.529737 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.529739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.538684 (+ 8945us) server_negotiation.cc:407] Connection header received
0807 08:41:59.538798 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.538799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.538811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.538842 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.539614 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.540556 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.547313 (+ 6757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.548277 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.548366 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.551306 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.551308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.551308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.551315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.553099 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.553621 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.553622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.553623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.554750 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.555014 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.555015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.555015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.555142 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.555666 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.555705 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.555778 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:41:59.556529 12225 mvcc.cc:205] Tried to move back new op lower bound from 7315833321667252224 to 7315833321589211136. Current Snapshot: MvccSnapshot[applied={T|T < 7315833321667252224}]
Aug 07 08:41:59 dist-test-slave-h6n0 krb5kdc[11838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092119, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:41:59.559821 12221 catalog_manager.cc:1347] Generated new cluster ID: 2257353001f64e3198a3b69bfa53fa86
I20260807 08:41:59.559934 12221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:41:59.563136 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.554166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:42325 (local address 127.0.0.1:38818)
0807 08:41:59.554225 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 08:41:59.554235 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:41:59.554269 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:41:59.554506 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:41:59.554507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:41:59.554733 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:41:59.554792 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.554794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.555787 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.555788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.556644 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:41:59.556646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.557343 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.557345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:41:59.557433 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.557927 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:41:59.557932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:41:59.559626 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:41:59.561850 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.561852 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.561853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.562099 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.562356 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:41:59.562357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:41:59.562357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:41:59.562429 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:41:59.562784 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:41:59.562785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:41:59.562847 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 08:41:59.562912 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:41:59.563023 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 08:41:59.563565 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:41:59.554344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38818 (local address 127.10.207.60:42325)
0807 08:41:59.554383 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.554385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.554392 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:41:59.554411 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.554412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.554421 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.554476 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.554899 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.555661 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.556886 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:41:59.557201 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:41:59.557261 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:41:59.559735 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:41:59.559737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:41:59.559737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:41:59.559744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:41:59.561706 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.562197 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.562199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.562199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.562272 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:41:59.562504 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:41:59.562505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:41:59.562505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:41:59.562673 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:41:59.562740 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:41:59.563386 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 08:41:59.563470 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:41:59.565802 12061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: Bootstrap starting.
I20260807 08:41:59.566735 12061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: Neither blocks nor log segments found. Creating new log.
I20260807 08:41:59.567073 12061 log.cc:824] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:41:59.583618 12061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: No bootstrap required, opened a new log
I20260807 08:41:59.583979 12221 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:41:59.584764 12221 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:41:59.586176 12061 raft_consensus.cc:348] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.586361 12061 raft_consensus.cc:729] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61e4413174047a280837d66503c989f, State: Initialized, Role: FOLLOWER
I20260807 08:41:59.586460 12061 consensus_queue.cc:260] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } }
I20260807 08:41:59.586913 12061 sys_catalog.cc:564] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:41:59.587013 12018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:41:59.588305 12018 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [term 1 FOLLOWER]: Refusing update from remote peer e44c49549bf74fe8a8cf78ccf4e63176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:41:59.586858 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.588836 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:41:59.588531 12020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d61e4413174047a280837d66503c989f"
I20260807 08:41:59.589073 12191 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:41:59.590452 12020 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44c49549bf74fe8a8cf78ccf4e63176 in term 1.
W20260807 08:41:59.591149 12247 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:41:59.591228 12247 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:41:59.592523 12221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176: Generated new TSK 0
I20260807 08:41:59.592829 12221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:41:59.593065 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e44c49549bf74fe8a8cf78ccf4e63176. Latest consensus state: current_term: 1 leader_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.593238 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:41:59.593317 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.593411 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:41:59.593827 12250 mvcc.cc:205] Tried to move back new op lower bound from 7315833321809027072 to 7315833321589211136. Current Snapshot: MvccSnapshot[applied={T|T < 7315833321667252224}]
I20260807 08:41:59.599502 12208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44c49549bf74fe8a8cf78ccf4e63176" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 41611 } } peers { permanent_uuid: "d61e4413174047a280837d66503c989f" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 39917 } } peers { permanent_uuid: "a0140423275541afb386463a5d78a76d" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 42325 } } }
I20260807 08:41:59.599614 12208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44c49549bf74fe8a8cf78ccf4e63176 [sys.catalog]: This master's current role is: LEADER
W20260807 08:41:59.631448 11953 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.629513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.10.207.62:41611)
0807 08:41:59.629565 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.629567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.630698 (+ 1131us) server_negotiation.cc:407] Connection header received
0807 08:41:59.630838 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.630839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.630851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.630887 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.631374 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57168: BlockingRecv error: recv got EOF from 127.0.0.1:57168 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 08:41:59.632297 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.629800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.10.207.61:39917)
0807 08:41:59.629845 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.629847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.631719 (+ 1872us) server_negotiation.cc:407] Connection header received
0807 08:41:59.631842 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.631843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.631856 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.631889 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.632235 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32810: BlockingRecv error: recv got EOF from 127.0.0.1:32810 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 08:41:59.633153 12169 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.630042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38830 (local address 127.10.207.60:42325)
0807 08:41:59.630090 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.630092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.632567 (+ 2475us) server_negotiation.cc:407] Connection header received
0807 08:41:59.632681 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.632682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.632695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.632729 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.633090 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38830: BlockingRecv error: recv got EOF from 127.0.0.1:38830 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 08:41:59.635900 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.635051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32826 (local address 127.10.207.61:39917)
0807 08:41:59.635090 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.635092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.635326 (+ 234us) server_negotiation.cc:407] Connection header received
0807 08:41:59.635443 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.635444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.635462 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.635496 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.635857 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32826: BlockingRecv error: recv got EOF from 127.0.0.1:32826 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 08:41:59.639137 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.638236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32842 (local address 127.10.207.61:39917)
0807 08:41:59.638271 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.638273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.638595 (+ 322us) server_negotiation.cc:407] Connection header received
0807 08:41:59.638708 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.638709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.638719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.638752 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.639097 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32842: BlockingRecv error: recv got EOF from 127.0.0.1:32842 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 08:41:59.642999 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.642229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.10.207.61:39917)
0807 08:41:59.642261 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.642263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.642478 (+ 215us) server_negotiation.cc:407] Connection header received
0807 08:41:59.642612 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.642613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.642623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.642652 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.642962 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32848: BlockingRecv error: recv got EOF from 127.0.0.1:32848 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 08:41:59.648873 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.648124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.10.207.61:39917)
0807 08:41:59.648158 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.648159 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.648391 (+ 232us) server_negotiation.cc:407] Connection header received
0807 08:41:59.648486 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.648487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.648496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.648524 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.648823 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32858: BlockingRecv error: recv got EOF from 127.0.0.1:32858 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 08:41:59.658751 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.657952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32874 (local address 127.10.207.61:39917)
0807 08:41:59.657992 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.657993 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.658232 (+ 239us) server_negotiation.cc:407] Connection header received
0807 08:41:59.658327 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.658328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.658338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.658368 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.658709 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32874: BlockingRecv error: recv got EOF from 127.0.0.1:32874 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 08:41:59.676863 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.676004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.10.207.61:39917)
0807 08:41:59.676043 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.676045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.676300 (+ 255us) server_negotiation.cc:407] Connection header received
0807 08:41:59.676415 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.676416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.676427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.676458 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.676811 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32878: BlockingRecv error: recv got EOF from 127.0.0.1:32878 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:41:59.711207 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.710197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.10.207.61:39917)
0807 08:41:59.710241 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.710243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.710523 (+ 280us) server_negotiation.cc:407] Connection header received
0807 08:41:59.710652 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.710653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.710664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.710693 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.711129 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32888: BlockingRecv error: recv got EOF from 127.0.0.1:32888 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 08:41:59.777448 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.776714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32894 (local address 127.10.207.61:39917)
0807 08:41:59.776815 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.776818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.776829 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:41:59.776918 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.776919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.776931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.777004 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.777358 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32894: BlockingRecv error: recv got EOF from 127.0.0.1:32894 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 08:41:59.908142 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:41:59.907130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.10.207.61:39917)
0807 08:41:59.907221 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 08:41:59.907224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:41:59.907375 (+ 151us) server_negotiation.cc:407] Connection header received
0807 08:41:59.907491 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:41:59.907493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:41:59.907517 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:41:59.907609 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:41:59.908018 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32900: BlockingRecv error: recv got EOF from 127.0.0.1:32900 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 08:42:00.166347 12071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:00.165593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32904 (local address 127.10.207.61:39917)
0807 08:42:00.165698 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:00.165700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:00.165717 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:42:00.165791 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:00.165792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:00.165804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:00.165894 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:00.166256 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32904: BlockingRecv error: recv got EOF from 127.0.0.1:32904 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:42:00.521438 12188 catalog_manager.cc:1259] Loaded cluster ID: 2257353001f64e3198a3b69bfa53fa86
I20260807 08:42:00.521536 12188 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: loading cluster ID for follower catalog manager: success
I20260807 08:42:00.522397 12188 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: acquiring CA information for follower catalog manager: success
I20260807 08:42:00.522782 12188 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a0140423275541afb386463a5d78a76d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:42:00.591813 12247 catalog_manager.cc:1259] Loaded cluster ID: 2257353001f64e3198a3b69bfa53fa86
I20260807 08:42:00.591913 12247 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: loading cluster ID for follower catalog manager: success
I20260807 08:42:00.592773 12247 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: acquiring CA information for follower catalog manager: success
I20260807 08:42:00.593073 12247 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d61e4413174047a280837d66503c989f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:42:00.682891 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:00.679939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32918 (local address 127.10.207.61:39917)
0807 08:42:00.680086 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:00.680088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:00.680131 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:42:00.680232 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:00.680233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:00.680291 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:00.680377 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:00.680547 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.681052 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.682002 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.682219 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.682297 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:00.682501 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:00.682636 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:00.682705 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:00.682768 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 08:42:00.682786 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260807 08:42:00.688838 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:00.685122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38832 (local address 127.10.207.60:42325)
0807 08:42:00.685272 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:00.685274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:00.685319 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:42:00.685410 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:00.685411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:00.685455 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:00.685523 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:00.685609 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.686282 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.687513 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.687774 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.687848 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:00.688097 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:00.688375 (+ 278us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:00.688505 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:00.688722 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 08:42:00.688751 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260807 08:42:00.694285 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:00.691204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.10.207.62:41611)
0807 08:42:00.691342 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:00.691344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:00.691351 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:42:00.691399 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:00.691400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:00.691454 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:00.691686 (+ 232us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:00.691728 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.692516 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.693409 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.693639 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.693703 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:00.693931 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:00.694040 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:00.694102 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:00.694164 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:42:00.694185 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260807 08:42:00.694591 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:00.690972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.10.207.61:39917)
0807 08:42:00.691027 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:00.691031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:00.691294 (+ 263us) server_negotiation.cc:407] Connection header received
0807 08:42:00.691405 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:00.691407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:00.691418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:00.691486 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:00.691720 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.692409 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.693613 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.693922 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.693996 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:00.694156 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:00.694198 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:00.694231 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:00.694472 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 08:42:00.694502 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:42:00.694983 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:00.691452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.10.207.60:42325)
0807 08:42:00.691505 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:00.691507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:00.691671 (+ 164us) server_negotiation.cc:407] Connection header received
0807 08:42:00.691764 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:00.691765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:00.691775 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:00.691844 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:00.692023 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.692710 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.693924 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:00.694268 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:00.694340 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:00.694591 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:00.694629 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:00.694700 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:00.694831 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 08:42:00.694869 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:42:00.696549 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11855
I20260807 08:42:00.708665 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 11961
I20260807 08:42:00.721554 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12074
2026-08-07T08:42:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3095 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:42:00 dist-test-slave-h6n0 krb5kdc[12373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:42:00 dist-test-slave-h6n0 krb5kdc[12373](info): set up 2 sockets
Aug 07 08:42:00 dist-test-slave-h6n0 krb5kdc[12373](info): commencing operation
krb5kdc: starting...
W20260807 08:42:01.982460 11068 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/test-user.keytab.
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:42:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
Entry for principal kudu/127.10.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab.
I20260807 08:42:02.113512 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44403
--webserver_interface=127.10.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:44403,127.10.207.61:43959,127.10.207.60:38343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:42:02.205664 12391 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:42:02.205931 12391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:42:02.205950 12391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:42:02.205976 12391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:42:02.207355 12391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:42:02.207406 12391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:42:02.207422 12391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:42:02.207437 12391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:42:02.207450 12391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:42:02.209180 12391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:44403,127.10.207.61:43959,127.10.207.60:38343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.62.keytab
--principal=kudu/127.10.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.62:44403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:42:02.209508 12391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:42:02.209795 12391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:42:02.215498 12406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:42:02.215538 12400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:42:02.215760 12391 server_base.cc:1034] running on GCE node
W20260807 08:42:02.215498 12401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:42:02.216202 12391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:42:02.216567 12391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:42:02.217725 12391 hybrid_clock.cc:648] HybridClock initialized: now 1786092122217706 us; error 37 us; skew 500 ppm
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:42:02.220781 12391 init.cc:377] Logged in from keytab as kudu/127.10.207.62@KRBTEST.COM (short username kudu)
I20260807 08:42:02.221328 12391 webserver.cc:460] Webserver started at http://127.10.207.62:38937/ using document root <none> and password file <none>
I20260807 08:42:02.221566 12391 fs_manager.cc:359] Metadata directory not provided
I20260807 08:42:02.221616 12391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:42:02.221717 12391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:42:02.222950 12391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data/instance:
uuid: "cc2ab490f66a48a2b895c8b5c4324adc"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "3ae355c9cb058e0247e32459697ad57b"
server_key_iv: "982962ef769315cb4102e835d3e405ec"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.223321 12391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/wal/instance:
uuid: "cc2ab490f66a48a2b895c8b5c4324adc"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "3ae355c9cb058e0247e32459697ad57b"
server_key_iv: "982962ef769315cb4102e835d3e405ec"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.224758 12391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:42:02.225680 12416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:42:02.225958 12391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:42:02.226081 12391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/wal
uuid: "cc2ab490f66a48a2b895c8b5c4324adc"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "3ae355c9cb058e0247e32459697ad57b"
server_key_iv: "982962ef769315cb4102e835d3e405ec"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.226197 12391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:42:02.246176 12391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:42:02.248618 12391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:42:02.248799 12391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:42:02.254108 12391 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.62:44403
I20260807 08:42:02.254163 12491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.62:44403 every 8 connection(s)
I20260807 08:42:02.254658 12391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/data/info.pb
I20260807 08:42:02.255825 12492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:42:02.257534 12492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:42:02.259596 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12391
I20260807 08:42:02.259703 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-0/wal/instance
I20260807 08:42:02.259902 11068 external_mini_cluster.cc:1428] Setting key 10c97fe3e12fa4286dc90e734350ff51
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:42:02.268514 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.258362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44403 (local address 127.0.0.1:39404)
0807 08:42:02.258624 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.258669 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.259140 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.259497 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.259501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.259922 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.260192 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.260199 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.261383 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.261385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.262311 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.262314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.262941 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.262942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.263012 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.263612 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.263621 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.265064 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.266991 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.266998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.267000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.267194 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.267447 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.267448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.267448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.267546 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.267900 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.267905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.267966 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.268101 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.268342 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":103,"thread_start_us":108,"threads_started":1}
I20260807 08:42:02.268517 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.258424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.10.207.62:44403)
0807 08:42:02.258680 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.258686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.258748 (+ 62us) server_negotiation.cc:407] Connection header received
0807 08:42:02.259304 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.259307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.259327 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.259423 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.260320 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.261265 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.262412 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.262816 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.262961 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.265149 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.265157 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.265157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.265164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.266877 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.267290 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.267293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.267293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.267375 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.267614 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.267615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.267615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.267819 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.267908 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.268141 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.268275 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":141,"threads_started":1}
WARNING: no policy specified for kudu/127.10.207.61@KRBTEST.COM; defaulting to no policy
I20260807 08:42:02.269765 12492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:42:02.270128 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.270010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53338)
0807 08:42:02.270064 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.270109 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 08:42:02.270328 12421 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
I20260807 08:42:02.271006 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.270898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53348)
0807 08:42:02.270944 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.270982 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 08:42:02.271168 12492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:43959: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab.
Entry for principal kudu/127.10.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab.
I20260807 08:42:02.291235 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:43959
--webserver_interface=127.10.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab
--principal=kudu/127.10.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:44403,127.10.207.61:43959,127.10.207.60:38343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:42:02.320505 12492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } attempt: 1
I20260807 08:42:02.321209 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.321002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53362)
0807 08:42:02.321092 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.321158 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 08:42:02.321741 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.321629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53372)
0807 08:42:02.321685 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.321719 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 08:42:02.321875 12492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:43959: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
W20260807 08:42:02.377502 12502 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:42:02.377705 12502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:42:02.377722 12502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:42:02.377743 12502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:42:02.378863 12502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:42:02.378907 12502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:42:02.378922 12502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:42:02.378937 12502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:42:02.378949 12502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:42:02.380446 12502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:44403,127.10.207.61:43959,127.10.207.60:38343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.61.keytab
--principal=kudu/127.10.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.61:43959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:42:02.380702 12502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:42:02.380932 12502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:42:02.386031 12511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:42:02.386092 12509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:42:02.386114 12513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:42:02.386605 12502 server_base.cc:1034] running on GCE node
I20260807 08:42:02.386830 12502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:42:02.387066 12502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:42:02.388211 12502 hybrid_clock.cc:648] HybridClock initialized: now 1786092122388182 us; error 57 us; skew 500 ppm
I20260807 08:42:02.390134 12492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } attempt: 2
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:42:02.390700 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.390550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53382)
0807 08:42:02.390623 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.390665 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 08:42:02.391237 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.391101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53392)
0807 08:42:02.391172 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.391207 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 08:42:02.391422 12492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.61:43959: Network error: Client connection negotiation failed: client connection to 127.10.207.61:43959: connect: Connection refused (error 111)
I20260807 08:42:02.391518 12502 init.cc:377] Logged in from keytab as kudu/127.10.207.61@KRBTEST.COM (short username kudu)
I20260807 08:42:02.392112 12502 webserver.cc:460] Webserver started at http://127.10.207.61:34467/ using document root <none> and password file <none>
I20260807 08:42:02.392417 12502 fs_manager.cc:359] Metadata directory not provided
I20260807 08:42:02.392472 12502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:42:02.392652 12502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:42:02.393910 12502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data/instance:
uuid: "14545d6be07a429395ee651b72742361"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "dc12230c3f3b5ffb605a1fa90ceca5fb"
server_key_iv: "09592d8506eb9242bd4e0a7ad627b108"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.394380 12502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/wal/instance:
uuid: "14545d6be07a429395ee651b72742361"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "dc12230c3f3b5ffb605a1fa90ceca5fb"
server_key_iv: "09592d8506eb9242bd4e0a7ad627b108"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.395980 12502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:42:02.396925 12523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:42:02.397188 12502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:42:02.397312 12502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/wal
uuid: "14545d6be07a429395ee651b72742361"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "dc12230c3f3b5ffb605a1fa90ceca5fb"
server_key_iv: "09592d8506eb9242bd4e0a7ad627b108"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.397399 12502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:42:02.420660 12502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:42:02.423478 12502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:42:02.423671 12502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:42:02.428750 12502 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.61:43959
I20260807 08:42:02.428829 12596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.61:43959 every 8 connection(s)
I20260807 08:42:02.429324 12502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/data/info.pb
I20260807 08:42:02.430583 12597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:42:02.432214 12597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:42:02.438211 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12502
I20260807 08:42:02.438329 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-1/wal/instance
I20260807 08:42:02.438611 11068 external_mini_cluster.cc:1428] Setting key f6380926151175d14a70358326c68fd1
I20260807 08:42:02.442822 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.432899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44403 (local address 127.0.0.1:39420)
0807 08:42:02.433088 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.433130 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.433590 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.433910 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.433913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.434164 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.434376 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.434388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.435201 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.435202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.436022 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.436024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.436440 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.436440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.436543 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.437069 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.437077 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.438886 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.440846 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.440855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.440858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.441243 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.441481 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.441482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.441482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.441654 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.442001 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.442010 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.442256 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.442401 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.442622 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260807 08:42:02.442900 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.433003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.10.207.62:44403)
0807 08:42:02.433062 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.433068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.433231 (+ 163us) server_negotiation.cc:407] Connection header received
0807 08:42:02.433723 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.433725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.433741 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.433832 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.434475 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.435099 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.436111 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.436337 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.436402 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.439016 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.439018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.439019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.439028 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.440740 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.441344 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.441345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.441345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.441399 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.441765 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.441766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.441766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.441900 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.441979 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.442474 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.442630 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:42:02.443857 12597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.207.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.207.60@KRBTEST.COM" created.
I20260807 08:42:02.452186 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.444300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53406)
0807 08:42:02.444365 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.444378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.444429 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.444701 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.444702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.444867 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.444923 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.444925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.445894 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.445895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.446416 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.446417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.447013 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.447014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.447085 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.447568 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.447574 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.448902 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.450902 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.450903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.450903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.451096 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.451350 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.451351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.451352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.451435 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.451804 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.451805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.451852 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.451914 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.452008 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 08:42:02.452199 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.444344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53406 (local address 127.10.207.61:43959)
0807 08:42:02.444540 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.444545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.444570 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:42:02.444590 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.444592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.444604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.444702 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.445004 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.445801 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.446514 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.446916 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.447013 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.448991 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.449003 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.449003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.449012 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.450806 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.451213 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.451216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.451216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.451275 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.451512 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.451513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.451514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.451725 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.451817 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.451972 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.452082 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":134,"threads_started":1}
I20260807 08:42:02.453068 12597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:42:02.453495 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.453333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42410)
0807 08:42:02.453397 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.453461 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:42:02.453711 12529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
I20260807 08:42:02.454218 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.454094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42412)
0807 08:42:02.454152 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.454187 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 08:42:02.454352 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38343: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab.
Entry for principal kudu/127.10.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab.
I20260807 08:42:02.469388 11068 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDqD6RB/build/release/bin/kudu
/tmp/dist-test-taskDqD6RB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:38343
--webserver_interface=127.10.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab
--principal=kudu/127.10.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.207.20:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.207.62:44403,127.10.207.61:43959,127.10.207.60:38343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:42:02.482607 12492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } attempt: 3
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:42:02.490829 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.483135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.10.207.61:43959)
0807 08:42:02.483206 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.483208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.483218 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:42:02.483289 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.483290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.483299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.483349 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.483830 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.484411 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.485562 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.485783 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.485854 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.488196 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.488197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.488198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.488206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.489660 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.490014 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.490016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.490016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.490075 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.490281 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.490282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.490282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.490388 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.490462 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.490661 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.490738 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:42:02.490835 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.483056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53408)
0807 08:42:02.483147 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.483160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.483212 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.483394 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.483395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.483614 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.483717 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.483720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.484505 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.484507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.485455 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.485457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.485873 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.485874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.485962 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.486469 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.486475 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.488097 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.489754 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.489755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.489755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.489937 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.490153 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.490154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.490154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.490214 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.490468 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.490470 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.490522 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.490637 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.490745 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 08:42:02.491423 12492 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:42:02.491833 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.491729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42414)
0807 08:42:02.491782 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.491810 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 08:42:02.492321 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.492238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42424)
0807 08:42:02.492281 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.492307 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 08:42:02.492443 12492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38343: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
I20260807 08:42:02.503573 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } attempt: 1
I20260807 08:42:02.504061 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.503927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42430)
0807 08:42:02.504000 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.504039 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 08:42:02.504488 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.504392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42432)
0807 08:42:02.504434 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.504464 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 08:42:02.504705 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38343: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
I20260807 08:42:02.523593 12492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } attempt: 1
I20260807 08:42:02.524024 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.523897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42440)
0807 08:42:02.523967 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.524002 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 08:42:02.524366 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.524263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42446)
0807 08:42:02.524310 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.524335 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 08:42:02.524463 12492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38343: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
W20260807 08:42:02.568871 12606 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:42:02.569171 12606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:42:02.569192 12606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:42:02.569252 12606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:42:02.570578 12606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:42:02.570624 12606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:42:02.570639 12606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:42:02.570653 12606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:42:02.570667 12606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:42:02.572351 12606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.207.20:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.207.62:44403,127.10.207.61:43959,127.10.207.60:38343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/krb5kdc/kudu_127.10.207.60.keytab
--principal=kudu/127.10.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.207.60:38343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:42:02.572680 12606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:42:02.572867 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } attempt: 2
I20260807 08:42:02.572984 12606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:42:02.573416 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.573278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42448)
0807 08:42:02.573350 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.573390 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 08:42:02.574036 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.573925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42462)
0807 08:42:02.573990 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.574019 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 08:42:02.574226 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38343: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
W20260807 08:42:02.578997 12615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:42:02.579006 12614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:42:02.579099 12606 server_base.cc:1034] running on GCE node
W20260807 08:42:02.578996 12617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:42:02.579521 12606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:42:02.579895 12606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:42:02.581053 12606 hybrid_clock.cc:648] HybridClock initialized: now 1786092122581036 us; error 44 us; skew 500 ppm
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:42:02.584065 12606 init.cc:377] Logged in from keytab as kudu/127.10.207.60@KRBTEST.COM (short username kudu)
I20260807 08:42:02.584558 12606 webserver.cc:460] Webserver started at http://127.10.207.60:37947/ using document root <none> and password file <none>
I20260807 08:42:02.584833 12606 fs_manager.cc:359] Metadata directory not provided
I20260807 08:42:02.584908 12606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:42:02.585070 12606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:42:02.586278 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data/instance:
uuid: "db11ba9dc0c1434a875fed3d167d9ecf"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "74712d388e3629794314099a54b18b69"
server_key_iv: "a6a61b58a761e7dfda0a0acd162be359"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.586735 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/wal/instance:
uuid: "db11ba9dc0c1434a875fed3d167d9ecf"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "74712d388e3629794314099a54b18b69"
server_key_iv: "a6a61b58a761e7dfda0a0acd162be359"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.588222 12606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:42:02.589051 12627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:42:02.589289 12606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:42:02.589408 12606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data,/tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/wal
uuid: "db11ba9dc0c1434a875fed3d167d9ecf"
format_stamp: "Formatted at 2026-08-07 08:42:02 on dist-test-slave-h6n0"
server_key: "74712d388e3629794314099a54b18b69"
server_key_iv: "a6a61b58a761e7dfda0a0acd162be359"
server_key_version: "encryptionkey@0"
I20260807 08:42:02.589488 12606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:42:02.599611 12492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } attempt: 2
I20260807 08:42:02.600029 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.599910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42474)
0807 08:42:02.599970 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.600005 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:42:02.600409 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.600300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42484)
0807 08:42:02.600350 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.600381 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 08:42:02.600531 12492 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.207.60:38343: Network error: Client connection negotiation failed: client connection to 127.10.207.60:38343: connect: Connection refused (error 111)
I20260807 08:42:02.601094 12606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:42:02.603765 12606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:42:02.603947 12606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:42:02.608892 12606 rpc_server.cc:307] RPC server started. Bound to: 127.10.207.60:38343
I20260807 08:42:02.608949 12697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.207.60:38343 every 8 connection(s)
I20260807 08:42:02.609345 12606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/data/info.pb
I20260807 08:42:02.610423 12698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:42:02.612702 12698 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:42:02.615897 11068 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDqD6RB/build/release/bin/kudu as pid 12606
I20260807 08:42:02.616055 11068 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDqD6RB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091830136701-11068-0/minicluster-data/master-2/wal/instance
I20260807 08:42:02.616328 11068 external_mini_cluster.cc:1428] Setting key 5e5b0712a41c0353693e23b07e9ba143
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
I20260807 08:42:02.627432 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.613569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39428 (local address 127.10.207.62:44403)
0807 08:42:02.613631 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.613633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.614006 (+ 373us) server_negotiation.cc:407] Connection header received
0807 08:42:02.614562 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.614563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.614576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.614634 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.615259 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.615765 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.617107 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.617450 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.617533 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.620867 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.620868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.620869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.620876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.625059 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.625604 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.625606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.625607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.625691 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.626028 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.626029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.626030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.626224 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.626300 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.627238 (+ 938us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.627334 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:42:02.627871 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.613523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.62:44403 (local address 127.0.0.1:39428)
0807 08:42:02.613873 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.613917 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.614423 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.614703 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.614707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.614978 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.615167 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.615171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.615862 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.615863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.616954 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.616957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.617669 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.617672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.617840 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.618406 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.618419 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.620757 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.625165 (+ 4408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.625172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.625176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.625504 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.625779 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.625780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.625780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.625949 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.626895 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.626900 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.627044 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.627213 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.627389 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":100,"threads_started":1}
I20260807 08:42:02.628630 12698 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:42:02.633490 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.618770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42488 (local address 127.10.207.60:38343)
0807 08:42:02.622745 (+ 3975us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.622751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.622781 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:42:02.622809 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.622812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.622829 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.622938 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.623934 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.625155 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.626176 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.626702 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.626808 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.629837 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.629846 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.629846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.629854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.631924 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.632410 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.632414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.632415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.632479 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.632750 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.632751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.632751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.632939 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.633097 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.633242 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.633336 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3937,"thread_start_us":84,"threads_started":1}
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:42:02.637431 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.629002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.61:43959 (local address 127.0.0.1:53424)
0807 08:42:02.629042 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.629053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.629091 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.629316 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.629317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.629539 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.629601 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.629605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.630590 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.630591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.631409 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.631413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.631988 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.631989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.632065 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.632522 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.632527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.633966 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.636257 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.636257 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.636258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.636482 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.636695 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.636696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.636697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.636760 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.637070 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.637071 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.637119 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.637201 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.637314 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 08:42:02.637496 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.629089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.10.207.61:43959)
0807 08:42:02.629189 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.629192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.629200 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:42:02.629234 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.629235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.629245 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.629312 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.629696 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.630448 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.631517 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.631856 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.631904 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.634066 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.634068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.634068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.634076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.635968 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.636560 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.636561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.636562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.636623 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.636830 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.636831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.636831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.636979 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.637022 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.637272 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.637393 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:42:02.638623 12698 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.62@KRBTEST.COM
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.60@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:42:02.644801 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.634485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39432 (local address 127.10.207.62:44403)
0807 08:42:02.634576 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.634578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.634587 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:42:02.634612 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.634613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.634623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.634675 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.636420 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.636926 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.637835 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.638124 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.638207 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.641038 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.641039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.641040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.641050 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.643255 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.643682 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.643683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.643683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.643743 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.643990 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.643991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.643991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.644147 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.644367 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.644581 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.644674 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:42:02.647126 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.638950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42492)
0807 08:42:02.639018 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.639030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.639074 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.639263 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.639264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.639420 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.639464 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.639466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.640277 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.640278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.641013 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.641016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.641554 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.641555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.641619 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.642136 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.642143 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.643714 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.646025 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.646026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.646026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.646248 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.646429 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.646430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.646430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.646479 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.646794 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.646794 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.646829 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.646893 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.647001 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 08:42:02.647609 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.638983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.10.207.60:38343)
0807 08:42:02.639023 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.639025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.639090 (+ 65us) server_negotiation.cc:407] Connection header received
0807 08:42:02.639152 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.639153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.639171 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.639242 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.639552 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.640205 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.641103 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.641469 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.641554 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.643801 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.643803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.643803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.643810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.645931 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.646321 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.646321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.646322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.646370 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.646571 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.646572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.646572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.646707 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.646771 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.647258 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.647347 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:42:02.649684 12698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: Bootstrap starting.
I20260807 08:42:02.651033 12698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: Neither blocks nor log segments found. Creating new log.
I20260807 08:42:02.651371 12698 log.cc:824] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.207.61@KRBTEST.COM
I20260807 08:42:02.652596 12698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: No bootstrap required, opened a new log
I20260807 08:42:02.655319 12698 raft_consensus.cc:348] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.655553 12698 raft_consensus.cc:729] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db11ba9dc0c1434a875fed3d167d9ecf, State: Initialized, Role: FOLLOWER
I20260807 08:42:02.655661 12698 consensus_queue.cc:260] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.655993 12706 sys_catalog.cc:455] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.656085 12698 sys_catalog.cc:564] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:42:02.656096 12706 sys_catalog.cc:458] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:42:02.657979 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.645659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.10.207.61:43959)
0807 08:42:02.645721 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.645723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.645735 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:42:02.645760 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.645761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.645773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.645853 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.646738 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.647603 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.648700 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.648944 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.649028 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.652010 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.652012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.652012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.652020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.656346 (+ 4326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.656942 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.656944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.656944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.657019 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.657244 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.657245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.657245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.657411 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.657711 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.657769 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.657866 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 08:42:02.658368 12722 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:42:02.658432 12722 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:42:02.660225 11068 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:42:02.661501 12451 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cc2ab490f66a48a2b895c8b5c4324adc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39432
I20260807 08:42:02.659368 12606 master_runner.cc:186] Error getting master registration for 127.10.207.62:44403: OK, instance_id { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" instance_seqno: 1786092122252635 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:42:02.663923 12606 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:42:02.665359 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } attempt: 3
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.61@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:42:02.675266 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.665635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42496)
0807 08:42:02.665680 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.665693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.665741 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.666082 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.666085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.666331 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.666408 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.666411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.667543 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.667544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.668547 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.668550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.669171 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.669172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.669251 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.669800 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.669806 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.671672 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.673856 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.673858 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.673859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.674106 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.674351 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.674352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.674353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.674421 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.674850 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.674851 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.674907 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.674981 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.675104 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 08:42:02.675722 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.665740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.10.207.60:38343)
0807 08:42:02.665829 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.665832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.665878 (+ 46us) server_negotiation.cc:407] Connection header received
0807 08:42:02.665910 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.665911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.665926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.666008 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.666511 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.667428 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.668664 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.669027 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.669097 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.671780 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.671781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.671782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.671790 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.673733 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.674209 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.674211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.674211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.674275 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.674494 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.674495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.674495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.674744 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.674817 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.675505 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.675600 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:42:02.677801 12597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: Bootstrap starting.
I20260807 08:42:02.678570 12597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: Neither blocks nor log segments found. Creating new log.
I20260807 08:42:02.678875 12597 log.cc:824] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: Log is configured to *not* fsync() on all Append() calls
I20260807 08:42:02.679997 12597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: No bootstrap required, opened a new log
I20260807 08:42:02.682024 12597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.682171 12597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14545d6be07a429395ee651b72742361, State: Initialized, Role: FOLLOWER
I20260807 08:42:02.682248 12597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.682605 12597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:42:02.682610 12725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.682794 12725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:42:02.684835 12739 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:42:02.684955 12739 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:42:02.684876 12502 master_runner.cc:186] Error getting master registration for 127.10.207.62:44403: OK, instance_id { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" instance_seqno: 1786092122252635 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:42:02.687743 12502 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:42:02.699687 12492 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } attempt: 3
Aug 07 08:42:02 dist-test-slave-h6n0 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.207.62@KRBTEST.COM for kudu/127.10.207.60@KRBTEST.COM
I20260807 08:42:02.708904 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.699986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.207.60:38343 (local address 127.0.0.1:42506)
0807 08:42:02.700038 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:42:02.700048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:42:02.700087 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:42:02.700342 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:42:02.700343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:42:02.700541 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:42:02.700596 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.700598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.701593 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.701594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.702749 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:42:02.702755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.703271 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.703272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:42:02.703351 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.703901 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:42:02.703906 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:42:02.705701 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:42:02.707754 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.707756 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.707757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.707991 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.708228 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:42:02.708229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:42:02.708229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:42:02.708290 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:42:02.708567 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:42:02.708568 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:42:02.708617 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 08:42:02.708680 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:42:02.708789 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:42:02.709551 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.700096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.10.207.60:38343)
0807 08:42:02.700152 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.700154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.700167 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:42:02.700209 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.700210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.700222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.700320 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.700694 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.701497 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.702861 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.703190 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.703268 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.705804 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.705806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.705806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.705814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.707654 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.708092 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.708093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.708094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.708150 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.708354 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.708355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.708355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.708483 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.708540 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.709369 (+ 829us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.709453 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:42:02.711499 12492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc: Bootstrap starting.
I20260807 08:42:02.712251 12492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc: Neither blocks nor log segments found. Creating new log.
I20260807 08:42:02.712565 12492 log.cc:824] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:42:02.713610 12492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc: No bootstrap required, opened a new log
I20260807 08:42:02.715018 12492 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.715155 12492 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc2ab490f66a48a2b895c8b5c4324adc, State: Initialized, Role: FOLLOWER
I20260807 08:42:02.715240 12492 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.715581 12492 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:42:02.715777 12451 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:42:02.715870 12451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:42:02.716988 12451 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.717388 12451 leader_election.cc:288] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [CANDIDATE]: Term 1 election: Requested vote from peers 14545d6be07a429395ee651b72742361 (127.10.207.61:43959), db11ba9dc0c1434a875fed3d167d9ecf (127.10.207.60:38343)
I20260807 08:42:02.717702 12559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "14545d6be07a429395ee651b72742361"
I20260807 08:42:02.717865 12559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:42:02.718833 12559 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc2ab490f66a48a2b895c8b5c4324adc in term 1.
I20260807 08:42:02.719148 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.719187 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "db11ba9dc0c1434a875fed3d167d9ecf"
I20260807 08:42:02.719323 12665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:42:02.719410 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:42:02.720254 12665 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc2ab490f66a48a2b895c8b5c4324adc in term 1.
I20260807 08:42:02.720499 12421 leader_election.cc:302] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14545d6be07a429395ee651b72742361, cc2ab490f66a48a2b895c8b5c4324adc; no voters:
I20260807 08:42:02.720718 12744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:42:02.720955 12744 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [term 1 LEADER]: Becoming Leader. State: Replica: cc2ab490f66a48a2b895c8b5c4324adc, State: Running, Role: LEADER
I20260807 08:42:02.721112 12744 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } }
I20260807 08:42:02.721760 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc2ab490f66a48a2b895c8b5c4324adc. Latest consensus state: current_term: 1 leader_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.721894 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: This master's current role is: LEADER
I20260807 08:42:02.722958 12391 master_runner.cc:263] No leader master found from master cc2ab490f66a48a2b895c8b5c4324adc
I20260807 08:42:02.723031 12391 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:42:02.729534 12767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:42:02.729604 12767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:42:02.733055 12771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:42:02.733472 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.721911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.10.207.62:44403)
0807 08:42:02.722072 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.722075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.722864 (+ 789us) server_negotiation.cc:407] Connection header received
0807 08:42:02.722977 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.722978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.722990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.723020 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.723921 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.724708 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.725762 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.726055 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.726099 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.727923 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.727924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.727925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.727932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.729723 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.730459 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.730461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.730461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.730520 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.731222 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.731223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.731224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.731365 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.731453 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.733275 (+ 1822us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.733370 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260807 08:42:02.733615 12771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:42:02.739912 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.722371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.10.207.61:43959)
0807 08:42:02.722421 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.722423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.729047 (+ 6624us) server_negotiation.cc:407] Connection header received
0807 08:42:02.729800 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.729801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.729813 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.729850 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.731666 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.732400 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.734895 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.735220 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.735262 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.736912 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.736914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.736914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.736922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.738663 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.739052 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.739053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.739053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.739109 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.739314 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.739315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.739315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.739446 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.739543 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.739742 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.739822 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:42:02.745679 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:02.722931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.10.207.60:38343)
0807 08:42:02.722985 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.722987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.730114 (+ 7127us) server_negotiation.cc:407] Connection header received
0807 08:42:02.730256 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.730257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.730268 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.730312 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.731173 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.731908 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.734414 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:02.734738 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:02.734791 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:02.742682 (+ 7891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:42:02.742684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:42:02.742685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:42:02.742695 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:42:02.744414 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.744822 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.744823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.744823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.744878 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:42:02.745063 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:42:02.745064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:42:02.745064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:42:02.745210 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:42:02.745301 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:02.745495 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:42:02.745577 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:42:02.747684 12665 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [term 1 FOLLOWER]: Refusing update from remote peer cc2ab490f66a48a2b895c8b5c4324adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:42:02.747783 12559 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [term 1 FOLLOWER]: Refusing update from remote peer cc2ab490f66a48a2b895c8b5c4324adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:42:02.748036 12744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:42:02.748193 12768 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:42:02.749609 12706 sys_catalog.cc:455] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc2ab490f66a48a2b895c8b5c4324adc. Latest consensus state: current_term: 1 leader_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.749715 12706 sys_catalog.cc:458] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:42:02.749868 12423 tablet.cc:2369] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:42:02.750053 12725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc2ab490f66a48a2b895c8b5c4324adc. Latest consensus state: current_term: 1 leader_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.750155 12725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:42:02.751396 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.751504 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: This master's current role is: LEADER
I20260807 08:42:02.751682 12778 mvcc.cc:205] Tried to move back new op lower bound from 7315833334770761728 to 7315833334667210752. Current Snapshot: MvccSnapshot[applied={T|T < 7315833334770761728}]
I20260807 08:42:02.751987 12725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.752075 12725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:42:02.752288 12783 mvcc.cc:205] Tried to move back new op lower bound from 7315833334770761728 to 7315833334667210752. Current Snapshot: MvccSnapshot[applied={T|T < 7315833334770761728}]
I20260807 08:42:02.756390 12771 catalog_manager.cc:1347] Generated new cluster ID: de195c25959a49478bf78df79751da61
I20260807 08:42:02.756458 12771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:42:02.767550 12768 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.767660 12768 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc [sys.catalog]: This master's current role is: LEADER
I20260807 08:42:02.768222 12706 sys_catalog.cc:455] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2ab490f66a48a2b895c8b5c4324adc" member_type: VOTER last_known_addr { host: "127.10.207.62" port: 44403 } } peers { permanent_uuid: "14545d6be07a429395ee651b72742361" member_type: VOTER last_known_addr { host: "127.10.207.61" port: 43959 } } peers { permanent_uuid: "db11ba9dc0c1434a875fed3d167d9ecf" member_type: VOTER last_known_addr { host: "127.10.207.60" port: 38343 } } }
I20260807 08:42:02.768316 12706 sys_catalog.cc:458] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:42:02.768672 12781 mvcc.cc:205] Tried to move back new op lower bound from 7315833334770761728 to 7315833334667210752. Current Snapshot: MvccSnapshot[applied={T|T < 7315833334770761728}]
I20260807 08:42:02.778214 12771 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:42:02.778950 12771 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:42:02.788841 12771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc2ab490f66a48a2b895c8b5c4324adc: Generated new TSK 0
I20260807 08:42:02.789014 12771 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:42:02.816107 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.814301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.10.207.61:43959)
0807 08:42:02.814349 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.814351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.815407 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 08:42:02.815528 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.815530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.815542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.815573 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.816041 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53454: BlockingRecv error: recv got EOF from 127.0.0.1:53454 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 08:42:02.817008 12495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.814716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.10.207.62:44403)
0807 08:42:02.814756 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.814758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.816480 (+ 1722us) server_negotiation.cc:407] Connection header received
0807 08:42:02.816596 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.816597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.816609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.816639 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.816950 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39456: BlockingRecv error: recv got EOF from 127.0.0.1:39456 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 08:42:02.817932 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.815036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.10.207.60:38343)
0807 08:42:02.815076 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.815078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.817234 (+ 2156us) server_negotiation.cc:407] Connection header received
0807 08:42:02.817486 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.817487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.817497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.817540 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.817876 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42538: BlockingRecv error: recv got EOF from 127.0.0.1:42538 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 08:42:02.820291 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.819205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53458 (local address 127.10.207.61:43959)
0807 08:42:02.819239 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.819240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.819788 (+ 548us) server_negotiation.cc:407] Connection header received
0807 08:42:02.819905 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.819906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.819923 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.819953 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.820252 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53458: BlockingRecv error: recv got EOF from 127.0.0.1:53458 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 08:42:02.823292 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.822480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.10.207.61:43959)
0807 08:42:02.822547 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.822549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.822799 (+ 250us) server_negotiation.cc:407] Connection header received
0807 08:42:02.822907 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.822908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.822918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.822946 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.823253 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53470: BlockingRecv error: recv got EOF from 127.0.0.1:53470 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:42:02.827050 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.826256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.10.207.61:43959)
0807 08:42:02.826289 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.826290 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.826566 (+ 276us) server_negotiation.cc:407] Connection header received
0807 08:42:02.826669 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.826670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.826680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.826711 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.827011 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53480: BlockingRecv error: recv got EOF from 127.0.0.1:53480 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:42:02.832672 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.831981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53496 (local address 127.10.207.61:43959)
0807 08:42:02.832013 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.832015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.832228 (+ 213us) server_negotiation.cc:407] Connection header received
0807 08:42:02.832311 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.832311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.832320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.832346 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.832633 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53496: BlockingRecv error: recv got EOF from 127.0.0.1:53496 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:42:02.842310 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.841626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.10.207.61:43959)
0807 08:42:02.841658 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.841660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.841872 (+ 212us) server_negotiation.cc:407] Connection header received
0807 08:42:02.841955 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.841956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.841964 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.841991 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.842266 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53502: BlockingRecv error: recv got EOF from 127.0.0.1:53502 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 08:42:02.860049 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.859370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.10.207.61:43959)
0807 08:42:02.859409 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.859411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.859542 (+ 131us) server_negotiation.cc:407] Connection header received
0807 08:42:02.859622 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.859623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.859632 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.859700 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.859996 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53506: BlockingRecv error: recv got EOF from 127.0.0.1:53506 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 08:42:02.894213 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.893331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.10.207.61:43959)
0807 08:42:02.893420 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.893423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.893581 (+ 158us) server_negotiation.cc:407] Connection header received
0807 08:42:02.893659 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.893660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.893675 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.893746 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.894152 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53512: BlockingRecv error: recv got EOF from 127.0.0.1:53512 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 08:42:02.960789 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:02.959834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.10.207.61:43959)
0807 08:42:02.959949 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:02.959953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:02.960125 (+ 172us) server_negotiation.cc:407] Connection header received
0807 08:42:02.960174 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:02.960176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:02.960190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:02.960272 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:02.960697 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53514: BlockingRecv error: recv got EOF from 127.0.0.1:53514 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 08:42:03.091719 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:03.090430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.10.207.61:43959)
0807 08:42:03.090609 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:03.090611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:03.090735 (+ 124us) server_negotiation.cc:407] Connection header received
0807 08:42:03.090855 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:03.090856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:03.090876 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:03.090948 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:03.091612 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53526: BlockingRecv error: recv got EOF from 127.0.0.1:53526 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260807 08:42:03.349864 12601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:42:03.349209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53536 (local address 127.10.207.61:43959)
0807 08:42:03.349281 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:03.349283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:03.349294 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:42:03.349369 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:03.349370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:03.349381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:03.349436 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:42:03.349812 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53536: BlockingRecv error: recv got EOF from 127.0.0.1:53536 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:42:03.658944 12722 catalog_manager.cc:1259] Loaded cluster ID: de195c25959a49478bf78df79751da61
I20260807 08:42:03.659034 12722 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: loading cluster ID for follower catalog manager: success
I20260807 08:42:03.659879 12722 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: acquiring CA information for follower catalog manager: success
I20260807 08:42:03.660216 12722 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db11ba9dc0c1434a875fed3d167d9ecf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:42:03.685477 12739 catalog_manager.cc:1259] Loaded cluster ID: de195c25959a49478bf78df79751da61
I20260807 08:42:03.685549 12739 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: loading cluster ID for follower catalog manager: success
I20260807 08:42:03.686385 12739 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: acquiring CA information for follower catalog manager: success
I20260807 08:42:03.686746 12739 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14545d6be07a429395ee651b72742361: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:42:03.866485 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:03.863369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53552 (local address 127.10.207.61:43959)
0807 08:42:03.863510 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:03.863512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:03.863523 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:42:03.863605 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:03.863606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:03.863662 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:03.863756 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:03.864042 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.864552 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.865423 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.865673 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.865741 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:03.866048 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:03.866169 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:03.866253 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:03.866386 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 08:42:03.866416 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":89,"threads_started":1}
I20260807 08:42:03.871695 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:03.868699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.10.207.60:38343)
0807 08:42:03.868834 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:03.868836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:03.868844 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:42:03.868881 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:03.868882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:03.868917 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:03.869005 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:03.869177 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.869895 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.870814 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.871022 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.871091 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:03.871257 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:03.871382 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:03.871445 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:03.871494 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:42:03.871523 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260807 08:42:03.876771 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:03.873973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.10.207.61:43959)
0807 08:42:03.874017 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:03.874019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:03.874186 (+ 167us) server_negotiation.cc:407] Connection header received
0807 08:42:03.874267 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:03.874269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:03.874285 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:03.874341 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:03.874579 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.875148 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.875959 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.876250 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.876335 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:03.876508 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:03.876541 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:03.876599 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:03.876682 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 08:42:03.876707 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:42:03.877806 12910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:03.873996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.10.207.62:44403)
0807 08:42:03.874352 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:03.874355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:03.874363 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:42:03.874416 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:03.874417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:03.874488 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:03.874593 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:03.874775 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.875682 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.876724 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.877157 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.877227 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:03.877437 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:03.877560 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:03.877640 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:03.877669 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 08:42:03.877703 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":105,"threads_started":1}
I20260807 08:42:03.877887 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:42:03.874308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.10.207.60:38343)
0807 08:42:03.874356 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:42:03.874358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:42:03.874550 (+ 192us) server_negotiation.cc:407] Connection header received
0807 08:42:03.874615 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:42:03.874616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:42:03.874631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:42:03.874687 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:42:03.874857 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.875678 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.877086 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:42:03.877407 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:42:03.877476 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:42:03.877607 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:42:03.877642 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:42:03.877699 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:42:03.877785 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 08:42:03.877810 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:42:03.879777 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12391
I20260807 08:42:03.895880 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12502
I20260807 08:42:03.909302 11068 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDqD6RB/build/release/bin/kudu with pid 12606
2026-08-07T08:42:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3189 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6284 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293769 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST